WildFly-26.1.2: Server Fail Over is not working
406 views
Skip to first unread message
vijen reddy
Mar 9, 2023, 11:14:28 AM3/9/23
to WildFly
Hi All,
I am having trouble finding the cause "Why server fail over is not working".
Here is the scenario:
My application is deployed on 2 standalone wildfly servers and configured clustering.
I deployed my application on both servers and are working fine and distributing user sessions.
So I wanted to test fail over. I did these steps:
After deployment... couple of users logged in to the application and 1st user on node1 and 2nd user is on node2.
Now I killed/stopped node1/server1 to test the failover & 1st user accessed the application, as soon as 1st user did some action he got logged out instead transferred to node2.
Here I am not able to find the cause why user was logged out instead transfered to another active node...
I really appreciate if one help me to find out the cause...
Thanks,
VJ
vijen reddy
Mar 9, 2023, 3:44:51 PM3/9/23
to WildFly
Anyone knows about setting fail over???
vijen reddy
Mar 9, 2023, 5:46:48 PM3/9/23
to WildFly
Any inputs please....
On Thursday, March 9, 2023 at 11:14:28 AM UTC-5 vijen reddy wrote:
Paul Ferraro
Mar 13, 2023, 4:11:46 PM3/13/23
to WildFly
Can you be more specific? How is your application configured to authenticate users?
How does your application associated a user's authentication state with the user's HttpSession?
vijen reddy
Mar 29, 2023, 6:18:24 PM3/29/23
to WildFly
Hi Paul,
Sorry for late response... Here is our the class where the user is authenticated:
Sorry for late response... Here is our the class where the user is authenticated:
/**s
* An { AuthenticationProvider} implementation that retrieves user details
* from an { UserDetailsService}.
*
* @author Ben Alex
*/
/*
* This class should be in SINGLETON Scope because this object is being referenced in webflow-config.xml AND Springs is trying to access this object from session before putting in session
* And while restarting the server getting the ERROR: "Error creating bean with name 'daoAuthenticationProviderCnvg': Scope 'session' is not active for the current thread"
*/
@Component("daoAuthenticationProviderCnvg")
public class DaoAuthenticationProviderCnvg implements AuthenticationProvider {
private static final Logger LOGGER = LoggerFactory.getLogger(DaoAuthenticationProviderCnvg.class);
/**
* <p> This method is being called/executed when user clicks Login button on the screen</p>
* <p>
* <li>As soon as user clicks the Login button then application queries and gets the User record with the UserName(Login) by calling
* { com.converge.qtrade.services.login.LoginServiceImpl#loadUserByUsername(String)} method.</li>
* <li> If the User Records exists then application queries USERS_VW by calling {@link com.converge.qtrade.services.login.LoginServiceImpl#decryptPassword(String)} method to get decrypted user password to compare with the text entered on the screen</li>
* <li> If the passwords(entered text on the screen & decrypted password from DB) match then calls the { com.converge.qtrade.services.login.LoginServiceImpl#userAuthenticationUpdate(String, String)} procedure call to update the NECXADM.USER_LOGIN_LOG_VW record with loginCount,sessionId... etc., <li/>
* <li> If the passwords(entered text on the screen & decrypted password from DB) DO NOT match then calls the { com.converge.qtrade.services.login.LoginServiceImpl#userAuthenticationUpdate(String, String)} procedure call to update the USERS_VW record with invalidLoginCount <li/>
* </p>
*/
@Override
public Authentication authenticate(Authentication arg0) throws AuthenticationException {
try {
if (StringUtils.isBlank(arg0.getName()) || StringUtils.isBlank(arg0.getCredentials().toString())) {
throw new BadCredentialsException("Bad Credentials");
} else {
QTradeLoggedInUser loadedUser = (QTradeLoggedInUser) QTradeServiceBeans.getLoginService().loadUserByUsername(arg0.getName());
if (loadedUser != null && loadedUser.getUserId() != null) {
String decryptedPassword = QTradeServiceBeans.getLoginService().getDecryptedPassword(arg0.getName());
if (!decryptedPassword.equalsIgnoreCase(arg0.getCredentials().toString())) {
userAuthenticationUpdate(arg0.getName(), "N");
} else {
userAuthenticationUpdate(arg0.getName(), "Y");
UserLoginLogVwBean loginInfoBean = new UserLoginLogVwBean();
GenericBuilder.copyBean(loadedUser, loginInfoBean);
loginInfoBean.setServer(AppUtils.getServerNode());
String sessionId = ((HttpServletRequest) FacesContext.getCurrentInstance().getExternalContext().getRequest()).getSession().getId();
loginInfoBean.setSessionId(sessionId); /** This is session id which will be created after successfully logged in **/
loginInfoBean.setApplication(QTradeAttributes.APP_NAME);
QTradeServiceBeans.getLoginService().updateLoginCountAndDate(loginInfoBean);
}
} else {
throw new UsernameNotFoundException("User not found");
}
return new UsernamePasswordAuthenticationToken(loadedUser, loadedUser.getPassword(),
loadedUser.getAuthorities());
}
} catch (UsernameNotFoundException e) {
throw new UsernameNotFoundException(e.getMessage(), e);
} catch (DBErrorException de) {
throw new DBErrorException(de.getMessage());
} catch (APPErrorException e) {
return null;
} catch (Exception e) {
throw new BadCredentialsException("Bad Credentials");
}
}
@Override
public boolean supports(Class<? extends Object> arg0) {
return arg0.equals(UsernamePasswordAuthenticationToken.class);
}
private void userAuthenticationUpdate(String userLogin, String loginSuccessInd) throws Exception {
String msg = QTradeServiceBeans.getLoginService().userAuthenticationUpdate(userLogin, loginSuccessInd);
if (StringUtils.isNotBlank(msg)) {
throw new DBErrorException(msg);
}
}
}
* An { AuthenticationProvider} implementation that retrieves user details
* from an { UserDetailsService}.
*
* @author Ben Alex
*/
/*
* This class should be in SINGLETON Scope because this object is being referenced in webflow-config.xml AND Springs is trying to access this object from session before putting in session
* And while restarting the server getting the ERROR: "Error creating bean with name 'daoAuthenticationProviderCnvg': Scope 'session' is not active for the current thread"
*/
@Component("daoAuthenticationProviderCnvg")
public class DaoAuthenticationProviderCnvg implements AuthenticationProvider {
private static final Logger LOGGER = LoggerFactory.getLogger(DaoAuthenticationProviderCnvg.class);
/**
* <p> This method is being called/executed when user clicks Login button on the screen</p>
* <p>
* <li>As soon as user clicks the Login button then application queries and gets the User record with the UserName(Login) by calling
* { com.converge.qtrade.services.login.LoginServiceImpl#loadUserByUsername(String)} method.</li>
* <li> If the User Records exists then application queries USERS_VW by calling {@link com.converge.qtrade.services.login.LoginServiceImpl#decryptPassword(String)} method to get decrypted user password to compare with the text entered on the screen</li>
* <li> If the passwords(entered text on the screen & decrypted password from DB) match then calls the { com.converge.qtrade.services.login.LoginServiceImpl#userAuthenticationUpdate(String, String)} procedure call to update the NECXADM.USER_LOGIN_LOG_VW record with loginCount,sessionId... etc., <li/>
* <li> If the passwords(entered text on the screen & decrypted password from DB) DO NOT match then calls the { com.converge.qtrade.services.login.LoginServiceImpl#userAuthenticationUpdate(String, String)} procedure call to update the USERS_VW record with invalidLoginCount <li/>
* </p>
*/
@Override
public Authentication authenticate(Authentication arg0) throws AuthenticationException {
try {
if (StringUtils.isBlank(arg0.getName()) || StringUtils.isBlank(arg0.getCredentials().toString())) {
throw new BadCredentialsException("Bad Credentials");
} else {
QTradeLoggedInUser loadedUser = (QTradeLoggedInUser) QTradeServiceBeans.getLoginService().loadUserByUsername(arg0.getName());
if (loadedUser != null && loadedUser.getUserId() != null) {
String decryptedPassword = QTradeServiceBeans.getLoginService().getDecryptedPassword(arg0.getName());
if (!decryptedPassword.equalsIgnoreCase(arg0.getCredentials().toString())) {
userAuthenticationUpdate(arg0.getName(), "N");
} else {
userAuthenticationUpdate(arg0.getName(), "Y");
UserLoginLogVwBean loginInfoBean = new UserLoginLogVwBean();
GenericBuilder.copyBean(loadedUser, loginInfoBean);
loginInfoBean.setServer(AppUtils.getServerNode());
String sessionId = ((HttpServletRequest) FacesContext.getCurrentInstance().getExternalContext().getRequest()).getSession().getId();
loginInfoBean.setSessionId(sessionId); /** This is session id which will be created after successfully logged in **/
loginInfoBean.setApplication(QTradeAttributes.APP_NAME);
QTradeServiceBeans.getLoginService().updateLoginCountAndDate(loginInfoBean);
}
} else {
throw new UsernameNotFoundException("User not found");
}
return new UsernamePasswordAuthenticationToken(loadedUser, loadedUser.getPassword(),
loadedUser.getAuthorities());
}
} catch (UsernameNotFoundException e) {
throw new UsernameNotFoundException(e.getMessage(), e);
} catch (DBErrorException de) {
throw new DBErrorException(de.getMessage());
} catch (APPErrorException e) {
return null;
} catch (Exception e) {
throw new BadCredentialsException("Bad Credentials");
}
}
@Override
public boolean supports(Class<? extends Object> arg0) {
return arg0.equals(UsernamePasswordAuthenticationToken.class);
}
private void userAuthenticationUpdate(String userLogin, String loginSuccessInd) throws Exception {
String msg = QTradeServiceBeans.getLoginService().userAuthenticationUpdate(userLogin, loginSuccessInd);
if (StringUtils.isNotBlank(msg)) {
throw new DBErrorException(msg);
}
}
}
And on each request, the user's session is validated using the below class:
import java.io.IOException;
import java.sql.Connection;
import java.sql.SQLException;
import java.sql.Statement;
import javax.servlet.FilterChain;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.apache.commons.lang3.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.jdbc.datasource.DriverManagerDataSource;
import org.springframework.security.core.context.SecurityContext;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.web.context.HttpSessionSecurityContextRepository;
import org.springframework.web.filter.OncePerRequestFilter;
import com.converge.framework.helper.ConnectionHelper;
import com.converge.framework.login.CnvgUser;
/**
* <p>The Class SessionTimeOutFilter validates the user's session.
* if user has timed out of session then it simply redirects it to
* log in page.</p>
*/
public class SessionTimeOutFilter extends OncePerRequestFilter {
/** The timeout page. */
private String timeoutPage = "login";
protected final Logger logger = LoggerFactory.getLogger(this.getClass());
/* (non-Javadoc)
* @see org.springframework.web.filter.OncePerRequestFilter#doFilterInternal(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, javax.servlet.FilterChain)
*/
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) throws ServletException, IOException {
if ((request instanceof HttpServletRequest) && (response instanceof HttpServletResponse)) {
HttpServletRequest httpServletRequest = (HttpServletRequest) request;
HttpServletResponse httpServletResponse = (HttpServletResponse) response;
/* is session expire control required for this request? */
if (isSessionControlRequiredForThisResource(httpServletRequest)) {
/* is session invalid? */
if (isSessionInvalid(httpServletRequest)) {
String timeoutUrl = httpServletRequest.getContextPath() + "/secure/" + getTimeoutPage();
/*
* String forgotPasswordLink =
* httpServletRequest.getContextPath() +
* "/forgotPassword.faces";
*/
String redirect = timeoutUrl;
/*
* if(httpServletRequest.getRequestURI() != null &&
* httpServletRequest
* .getRequestURI().equalsIgnoreCase(forgotPasswordLink)){
* redirect = forgotPasswordLink; }
*/
logger.info("Session is invalid! redirecting to timeoutpage : " + redirect);
httpServletResponse.sendRedirect(redirect);
return;
}
}
}
try {
filterChain.doFilter(request, response);
} catch (Exception e) {
e.printStackTrace();
HttpServletRequest req = (HttpServletRequest) request;
CnvgUser user = (CnvgUser) SecurityContextHolder.getContext().getAuthentication().getPrincipal();
try {
DriverManagerDataSource ds = ConnectionHelper.getTradeDataSource();
Connection conn = ds.getConnection();
Statement st = conn.createStatement();
st.executeUpdate("DELETE FROM NECXADM.APPLICATION_SESSIONS WHERE SESSION_ID = '" + req.getSession().getId().substring(0, req.getSession().getId().indexOf('.')) + "' AND LOGIN = '" + user.getUsername() + "'");
conn.close();
ds = null;
} catch (SQLException sqle) {
sqle.printStackTrace();
} catch (Exception e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
throw new org.springframework.beans.BeanInstantiationException(null, timeoutPage);
}
}
/**
* Checks if is session control required for this resource.
*
* @param httpServletRequest the http servlet request
* @return true, if is session control required for this resource
*/
private boolean isSessionControlRequiredForThisResource(HttpServletRequest httpServletRequest) {
String requestPath = httpServletRequest.getRequestURI();
String forgotPasswordLink = httpServletRequest.getContextPath() + "/secure/forgotpassword";
boolean controlRequired = (!StringUtils.contains(requestPath, getTimeoutPage()) && !StringUtils.contains(requestPath, forgotPasswordLink) && !StringUtils.contains(requestPath, "javax.faces.resource"));
return controlRequired;
}
/**
* Checks if is session invalid.
*
* @param httpServletRequest the http servlet request
* @return true, if is session invalid
*/
private boolean isSessionInvalid(HttpServletRequest httpServletRequest) {
SecurityContext context = (SecurityContext) httpServletRequest.getSession().getAttribute(HttpSessionSecurityContextRepository.SPRING_SECURITY_CONTEXT_KEY);
boolean sessionInValid;
if (context == null) {
sessionInValid = true;
} else {
sessionInValid = (httpServletRequest.getRequestedSessionId() != null) && !httpServletRequest.isRequestedSessionIdValid();
}
return sessionInValid;
}
/**
* Gets the timeout page.
*
* @return the timeout page
*/
public String getTimeoutPage() {
return timeoutPage;
}
/**
* Sets the timeout page.
*
* @param timeoutPage the new timeout page
*/
public void setTimeoutPage(String timeoutPage) {
this.timeoutPage = timeoutPage;
}
}
import java.sql.Connection;
import java.sql.SQLException;
import java.sql.Statement;
import javax.servlet.FilterChain;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.apache.commons.lang3.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.jdbc.datasource.DriverManagerDataSource;
import org.springframework.security.core.context.SecurityContext;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.web.context.HttpSessionSecurityContextRepository;
import org.springframework.web.filter.OncePerRequestFilter;
import com.converge.framework.helper.ConnectionHelper;
import com.converge.framework.login.CnvgUser;
/**
* <p>The Class SessionTimeOutFilter validates the user's session.
* if user has timed out of session then it simply redirects it to
* log in page.</p>
*/
public class SessionTimeOutFilter extends OncePerRequestFilter {
/** The timeout page. */
private String timeoutPage = "login";
protected final Logger logger = LoggerFactory.getLogger(this.getClass());
/* (non-Javadoc)
* @see org.springframework.web.filter.OncePerRequestFilter#doFilterInternal(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, javax.servlet.FilterChain)
*/
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) throws ServletException, IOException {
if ((request instanceof HttpServletRequest) && (response instanceof HttpServletResponse)) {
HttpServletRequest httpServletRequest = (HttpServletRequest) request;
HttpServletResponse httpServletResponse = (HttpServletResponse) response;
/* is session expire control required for this request? */
if (isSessionControlRequiredForThisResource(httpServletRequest)) {
/* is session invalid? */
if (isSessionInvalid(httpServletRequest)) {
String timeoutUrl = httpServletRequest.getContextPath() + "/secure/" + getTimeoutPage();
/*
* String forgotPasswordLink =
* httpServletRequest.getContextPath() +
* "/forgotPassword.faces";
*/
String redirect = timeoutUrl;
/*
* if(httpServletRequest.getRequestURI() != null &&
* httpServletRequest
* .getRequestURI().equalsIgnoreCase(forgotPasswordLink)){
* redirect = forgotPasswordLink; }
*/
logger.info("Session is invalid! redirecting to timeoutpage : " + redirect);
httpServletResponse.sendRedirect(redirect);
return;
}
}
}
try {
filterChain.doFilter(request, response);
} catch (Exception e) {
e.printStackTrace();
HttpServletRequest req = (HttpServletRequest) request;
CnvgUser user = (CnvgUser) SecurityContextHolder.getContext().getAuthentication().getPrincipal();
try {
DriverManagerDataSource ds = ConnectionHelper.getTradeDataSource();
Connection conn = ds.getConnection();
Statement st = conn.createStatement();
st.executeUpdate("DELETE FROM NECXADM.APPLICATION_SESSIONS WHERE SESSION_ID = '" + req.getSession().getId().substring(0, req.getSession().getId().indexOf('.')) + "' AND LOGIN = '" + user.getUsername() + "'");
conn.close();
ds = null;
} catch (SQLException sqle) {
sqle.printStackTrace();
} catch (Exception e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
throw new org.springframework.beans.BeanInstantiationException(null, timeoutPage);
}
}
/**
* Checks if is session control required for this resource.
*
* @param httpServletRequest the http servlet request
* @return true, if is session control required for this resource
*/
private boolean isSessionControlRequiredForThisResource(HttpServletRequest httpServletRequest) {
String requestPath = httpServletRequest.getRequestURI();
String forgotPasswordLink = httpServletRequest.getContextPath() + "/secure/forgotpassword";
boolean controlRequired = (!StringUtils.contains(requestPath, getTimeoutPage()) && !StringUtils.contains(requestPath, forgotPasswordLink) && !StringUtils.contains(requestPath, "javax.faces.resource"));
return controlRequired;
}
/**
* Checks if is session invalid.
*
* @param httpServletRequest the http servlet request
* @return true, if is session invalid
*/
private boolean isSessionInvalid(HttpServletRequest httpServletRequest) {
SecurityContext context = (SecurityContext) httpServletRequest.getSession().getAttribute(HttpSessionSecurityContextRepository.SPRING_SECURITY_CONTEXT_KEY);
boolean sessionInValid;
if (context == null) {
sessionInValid = true;
} else {
sessionInValid = (httpServletRequest.getRequestedSessionId() != null) && !httpServletRequest.isRequestedSessionIdValid();
}
return sessionInValid;
}
/**
* Gets the timeout page.
*
* @return the timeout page
*/
public String getTimeoutPage() {
return timeoutPage;
}
/**
* Sets the timeout page.
*
* @param timeoutPage the new timeout page
*/
public void setTimeoutPage(String timeoutPage) {
this.timeoutPage = timeoutPage;
}
}
I hope this will help you to understand my issue.
Here I am attaching my standalone-ha.xml & vhosts.conf (loadbalancer apache setting setting) as well for your reference...
Paul Ferraro
Mar 29, 2023, 6:42:28 PM3/29/23
to vijen reddy, WildFly
Can you attach your application's web.xml?
Can you also attach UserLoginLogVwBean ?
Does your application also use Spring Session?
> --
> You received this message because you are subscribed to a topic in the Google Groups "WildFly" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/wildfly/yhyO4sxG_Hs/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to wildfly+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/wildfly/c594f636-bec6-4bab-a1aa-5c88b8fd6af3n%40googlegroups.com.
Can you also attach UserLoginLogVwBean ?
Does your application also use Spring Session?
> You received this message because you are subscribed to a topic in the Google Groups "WildFly" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/wildfly/yhyO4sxG_Hs/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to wildfly+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/wildfly/c594f636-bec6-4bab-a1aa-5c88b8fd6af3n%40googlegroups.com.
vijen reddy
Mar 30, 2023, 11:39:03 AM3/30/23
to WildFly
On Wednesday, March 29, 2023 at 6:42:28 PM UTC-4 Paul Ferraro wrote:
Can you attach your application's web.xml?
Can you also attach UserLoginLogVwBean ?
Here I attached my UserLoginVwBean & my web.xml files
Does your application also use Spring Session?
I think YES
Paul Ferraro
Mar 30, 2023, 12:21:00 PM3/30/23
to WildFly
On Thursday, March 30, 2023 at 11:39:03 AM UTC-4 vijen reddy wrote:
On Wednesday, March 29, 2023 at 6:42:28 PM UTC-4 Paul Ferraro wrote:Can you attach your application's web.xml?
Can you also attach UserLoginLogVwBean ?Here I attached my UserLoginVwBean & my web.xml files
I don't see any problems here, although this:
<listener>
<listener-class>org.springframework.security.web.session.HttpSessionEventPublisher</listener-class>
</listener>
Suggests the use of Spring Session.
Does your application also use Spring Session?I think YES
OK - this is the source of your problem.
Spring Session registers a filter that wraps the HttpServletRequest/HttpServletResponse of a given request such that any calls to HttpServletRequest.getSession(...) executed after this filter within the request chain will return objects from Spring's SessionRepository, completely bypassing WildFly's session manager. This renders Spring Session completely incompatible with and disconnected from server-managed authentication and session management facilities, whose filters execute before the filter installed by Spring Session. This is an inherent problem with Spring, and is why its ecosystem is so viral.
In other words, your issue is not due to, nor can be addressed by, WildFly, since your SecurityContext and HttpSessions are ultimately managed by Spring.
So long as you are using Spring Session, you should omit <distributable/> from your web.xml, since this will configure WildFly to use a distributed session manager that will never contain any sessions.
Which Spring SessionRepository implementation are you using?
vijen reddy
Mar 30, 2023, 12:53:41 PM3/30/23
to WildFly
Thanks You very much Paul for quick response... I added my comments below
Thanks,
Vijen
On Thursday, March 30, 2023 at 12:21:00 PM UTC-4 Paul Ferraro wrote:
On Thursday, March 30, 2023 at 11:39:03 AM UTC-4 vijen reddy wrote:On Wednesday, March 29, 2023 at 6:42:28 PM UTC-4 Paul Ferraro wrote:Can you attach your application's web.xml?
Can you also attach UserLoginLogVwBean ?Here I attached my UserLoginVwBean & my web.xml filesI don't see any problems here, although this:<listener><listener-class>org.springframework.security.web.session.HttpSessionEventPublisher</listener-class></listener>Suggests the use of Spring Session.
Does your application also use Spring Session?I think YESOK - this is the source of your problem.Spring Session registers a filter that wraps the HttpServletRequest/HttpServletResponse of a given request such that any calls to HttpServletRequest.getSession(...) executed after this filter within the request chain will return objects from Spring's SessionRepository, completely bypassing WildFly's session manager. This renders Spring Session completely incompatible with and disconnected from server-managed authentication and session management facilities, whose filters execute before the filter installed by Spring Session. This is an inherent problem with Spring, and is why its ecosystem is so viral.In other words, your issue is not due to, nor can be addressed by, WildFly, since your SecurityContext and HttpSessions are ultimately managed by Spring.So long as you are using Spring Session, you should omit <distributable/> from your web.xml, since this will configure WildFly to use a distributed session manager that will never contain any sessions.
OK I will try it by removing <distributable/> from my web.xml and will let you know the outcome.
Which Spring SessionRepository implementation are you using?
I am not sure :( ... How and where can find in my application??
vijen reddy
Mar 30, 2023, 1:57:11 PM3/30/23
to WildFly
Hi Paul,
I tried by removing <distributable/> from web.xml file whatever you suggested and it didn't work.
Here I am attaching the Server.log files for both servers(one is shutdown and another still active) after stopping the one node/server.
Please go thru them and let me know if anything you find...
Please find the attachments...
Thanks,
Vijen
Paul Ferraro
Mar 30, 2023, 3:19:53 PM3/30/23
to WildFly
On Thursday, March 30, 2023 at 1:57:11 PM UTC-4 vijen reddy wrote:
Hi Paul,I tried by removing <distributable/> from web.xml file whatever you suggested and it didn't work.
What do you mean by "it didn't work"? I never suggested that this would solve your failover problem - only that the use of <distributable/> in web.xml is only used by WildFly's session manager. Spring Session has no access to this configuration.
I'm not sure if my previous message was clear - so I will restate:
The inability of a user's security context to be preserved following failover is not caused by WildFly or your WildFly configuration. Your application's use of Spring completely bypasses WildFly's authentication/session mechanisms.
In other words, I suggest redirecting your questions to the appropriate Spring forums.
Here I am attaching the Server.log files for both servers(one is shutdown and another still active) after stopping the one node/server.
These logs give no indication of anything relevant.
vijen reddy
Apr 3, 2023, 11:41:37 AM4/3/23
to WildFly
Hi Paul,
I don't think it's true... because I tried to deploy a simple project without any spring security... just simple HttpServelet
And the cluster is not working...
Please help me here with the new war file
Here are my simple java files as an attachments.
Note: I am using same settings.. In other words same configurations which I told before... but I deployed this new war file to test clustering...
Thanks,
VJ
Paul Ferraro
Apr 3, 2023, 12:29:22 PM4/3/23
to WildFly
I still have yet to figure out what you mean by "And the cluster is not working..."
How you are starting your server instances? Have you verified that your server instances are indeed visible to each other? This would be evident from the server logs.
vijen reddy
Apr 4, 2023, 8:16:37 AM4/4/23
to WildFly
Hi Paul,
I still have yet to figure out what you mean by "And the cluster is not working..." >>>> I mean the server fail over is not working(session replication on other node if one node is down)
How you are starting your server instances? >>>> We use start_stop_jboss.sh script (custom script) to start the server and I am attaching the script with this email
I did this by using CLI command and it is showing only one node on each cluster: Where as it should show 2 nodes
On Node1/Server1:
[standalone@localhost:9990 /] /subsystem=jgroups/channel=ee:read-attribute(name=view,include-defaults=true)
{
"outcome" => "success",
"result" => "[devqtradevm01|0] (1) [devqtradevm01]"
}
On Node2/Server2:
{
"outcome" => "success",
"result" => "[devqtradevm01|0] (1) [devqtradevm01]"
}
On Node2/Server2:
[standalone@localhost:9990 /] /subsystem=jgroups/channel=ee:read-attribute(name=view,include-defaults=true)
{
"outcome" => "success",
"result" => "[devqtradevm02|0] (1) [devqtradevm02]"
}
From server.log:
2023-04-03 13:41:21,397 [,,] INFO [org.infinispan.CLUSTER] ISPN000078: Starting JGroups channel ejb
2023-04-03 13:41:21,397 [,,] INFO [org.infinispan.CLUSTER] ISPN000078: Starting JGroups channel ejb
2023-04-03 13:41:21,399 [,,] DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] JGroups protocol stack: org.jgroups.fork.ForkProtocol(stats=true;ergonomics=true;id=0)
2023-04-03 13:41:21,400 [,,] INFO [org.infinispan.CLUSTER] ISPN000094: Received new cluster view for channel ejb: [devqtradevm01|0] (1) [devqtradevm01]
2023-04-03 13:41:21,401 [,,] DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] Joined: [devqtradevm01], Left: []
2023-04-03 13:41:21,403 [,,] INFO [org.infinispan.CLUSTER] ISPN000078: Starting JGroups channel ejb
2023-04-03 13:41:21,405 [,,] INFO [org.infinispan.CLUSTER] ISPN000079: Channel ejb local address is devqtradevm01, physical addresses are [10.101.16.21:55200]
2023-04-03 13:41:21,410 [,,] INFO [org.infinispan.CLUSTER] ISPN000078: Starting JGroups channel ejb
2023-04-03 13:41:21,413 [,,] DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] JGroups protocol stack: org.jgroups.fork.ForkProtocol(stats=true;ergonomics=true;id=0)
2023-04-03 13:41:21,413 [,,] INFO [org.infinispan.CLUSTER] ISPN000094: Received new cluster view for channel ejb: [devqtradevm01|0] (1) [devqtradevm01]
2023-04-03 13:41:21,413 [,,] DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] Joined: [devqtradevm01], Left: []
2023-04-03 13:41:21,415 [,,] DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] JGroups protocol stack: org.jgroups.fork.ForkProtocol(stats=true;ergonomics=true;id=0)
2023-04-03 13:41:21,415 [,,] DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] JGroups protocol stack: org.jgroups.fork.ForkProtocol(stats=true;ergonomics=true;id=0)
2023-04-03 13:41:21,415 [,,] INFO [org.infinispan.CLUSTER] ISPN000094: Received new cluster view for channel ejb: [devqtradevm01|0] (1) [devqtradevm01]
2023-04-03 13:41:21,415 [,,] INFO [org.infinispan.CLUSTER] ISPN000094: Received new cluster view for channel ejb: [devqtradevm01|0] (1) [devqtradevm01]
2023-04-03 13:41:21,397 [,,] INFO [org.infinispan.CLUSTER] ISPN000078: Starting JGroups channel ejb
2023-04-03 13:41:21,399 [,,] DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] JGroups protocol stack: org.jgroups.fork.ForkProtocol(stats=true;ergonomics=true;id=0)
2023-04-03 13:41:21,400 [,,] INFO [org.infinispan.CLUSTER] ISPN000094: Received new cluster view for channel ejb: [devqtradevm01|0] (1) [devqtradevm01]
2023-04-03 13:41:21,401 [,,] DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] Joined: [devqtradevm01], Left: []
2023-04-03 13:41:21,403 [,,] INFO [org.infinispan.CLUSTER] ISPN000078: Starting JGroups channel ejb
2023-04-03 13:41:21,405 [,,] INFO [org.infinispan.CLUSTER] ISPN000079: Channel ejb local address is devqtradevm01, physical addresses are [10.101.16.21:55200]
2023-04-03 13:41:21,410 [,,] INFO [org.infinispan.CLUSTER] ISPN000078: Starting JGroups channel ejb
2023-04-03 13:41:21,413 [,,] DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] JGroups protocol stack: org.jgroups.fork.ForkProtocol(stats=true;ergonomics=true;id=0)
2023-04-03 13:41:21,413 [,,] INFO [org.infinispan.CLUSTER] ISPN000094: Received new cluster view for channel ejb: [devqtradevm01|0] (1) [devqtradevm01]
2023-04-03 13:41:21,413 [,,] DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] Joined: [devqtradevm01], Left: []
2023-04-03 13:41:21,415 [,,] DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] JGroups protocol stack: org.jgroups.fork.ForkProtocol(stats=true;ergonomics=true;id=0)
2023-04-03 13:41:21,415 [,,] DEBUG [org.infinispan.remoting.transport.jgroups.JGroupsTransport] JGroups protocol stack: org.jgroups.fork.ForkProtocol(stats=true;ergonomics=true;id=0)
2023-04-03 13:41:21,415 [,,] INFO [org.infinispan.CLUSTER] ISPN000094: Received new cluster view for channel ejb: [devqtradevm01|0] (1) [devqtradevm01]
2023-04-03 13:41:21,415 [,,] INFO [org.infinispan.CLUSTER] ISPN000094: Received new cluster view for channel ejb: [devqtradevm01|0] (1) [devqtradevm01]
=================================================================================================================================================================
When I check my apache server(load balancer) mod_cluster, it is showing correctly:
Thanks,
VJ
Paul Ferraro
Apr 4, 2023, 9:58:08 AM4/4/23
to WildFly
Your logs indicate that your cluster members are not communicating with each other.
Given the physical addresses listed in your server log, I suspect you need to update the network interface on which your servers will listen for cluster messages.
By default, jgroups uses a socket-binding that is bound to the "private" interface. By default, this looks like:
i.e.
<interfaces>
<!-- ... -->
<interface name="private">
<inet-address value="${jboss.bind.address.private:127.0.0.1}"/>
</interface>
<!-- ... -->
</interfaces>
<interface name="private">
<inet-address value="${jboss.bind.address.private:127.0.0.1}"/>
</interface>
<!-- ... -->
</interfaces>
You can either override the -Djboss.bind.address.private system property, or update the interface name/address in the server configuration.
vijen reddy
Apr 4, 2023, 10:43:34 AM4/4/23
to WildFly
Thanks for your reply Paul... It will be really helpful if you give me the example... as we are already overriding the bind address in our start-stop script (which was attached in last email incase if did not see it)
where $PRIVATEIP is server ip address (Ex: 10.101.16.21)
Like this:
-Djboss.bind.address.private=$PRIVATEIP
where $PRIVATEIP is server ip address (Ex: 10.101.16.21)
Paul Ferraro
Apr 4, 2023, 10:56:48 AM4/4/23
to WildFly
OK - the next logical question is: how are your servers configured to discover one another? Can you attach your jgroups subsystem configuration?
vijen reddy
Apr 4, 2023, 1:37:38 PM4/4/23
to WildFly
Hi Paul,
FYI: My 2 servers individual standalone servers on 2 different VMs and each VM has it's own load balancing apache server.
Can you attach your jgroups subsystem configuration?
<channels default="ee">
<channel name="ee" stack="udp" cluster="ejb"/>
</channels>
<stacks>
<stack name="udp">
<transport type="UDP" socket-binding="jgroups-udp"/>
<protocol type="PING"/>
<protocol type="MERGE3"/>
<socket-protocol type="FD_SOCK" socket-binding="jgroups-udp-fd"/>
<protocol type="FD_ALL"/>
<protocol type="VERIFY_SUSPECT"/>
<protocol type="pbcast.NAKACK2"/>
<protocol type="UNICAST3"/>
<protocol type="pbcast.STABLE"/>
<protocol type="pbcast.GMS"/>
<protocol type="UFC"/>
<protocol type="MFC"/>
<protocol type="FRAG3"/>
</stack>
<stack name="tcp">
<transport type="TCP" socket-binding="jgroups-tcp"/>
<socket-protocol type="MPING" socket-binding="jgroups-mping"/>
<protocol type="MERGE3"/>
<socket-protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>
<protocol type="FD_ALL"/>
<protocol type="VERIFY_SUSPECT"/>
<protocol type="pbcast.NAKACK2"/>
<protocol type="UNICAST3"/>
<protocol type="pbcast.STABLE"/>
<protocol type="pbcast.GMS"/>
<protocol type="MFC"/>
<protocol type="FRAG3"/>
</stack>
</stacks>
</subsystem>
how are your servers configured to discover one another?
Honestly I don't know... but may be with this configuration:
<subsystem xmlns="urn:jboss:domain:modcluster:5.0">
<proxy name="default" advertise="false" advertise-socket="modcluster" balancer="devqtradewf" listener="ajp" proxies="proxy1 proxy2">
<dynamic-load-provider>
<load-metric type="cpu"/>
</dynamic-load-provider>
</proxy>
</subsystem>
<proxy name="default" advertise="false" advertise-socket="modcluster" balancer="devqtradewf" listener="ajp" proxies="proxy1 proxy2">
<dynamic-load-provider>
<load-metric type="cpu"/>
</dynamic-load-provider>
</proxy>
</subsystem>
....
....
<socket-binding-group ....
<outbound-socket-binding name="proxy1">
<remote-destination host="10.101.16.21" port="80"/>
</outbound-socket-binding>
<outbound-socket-binding name="proxy2">
<remote-destination host="10.101.16.22" port="80"/>
</outbound-socket-binding>
<remote-destination host="10.101.16.21" port="80"/>
</outbound-socket-binding>
<outbound-socket-binding name="proxy2">
<remote-destination host="10.101.16.22" port="80"/>
</outbound-socket-binding>
</socket-binding-group>
Paul Ferraro
Apr 4, 2023, 2:42:14 PM4/4/23
to WildFly
mod_cluster has nothing to do with cluster discovery.
You seem to be using the "udp" protocol stack, which relies on multicast via the address/port specified via the "jgroups-udp" socket binding to discover cluster members.
Have you verified that your VMs are capable of multicasting?
vijen reddy
Apr 4, 2023, 3:39:08 PM4/4/23
to WildFly
Yes Paul, I confirmed with our Network Admin and they are capable of multicasting
Paul Ferraro
Apr 4, 2023, 5:04:59 PM4/4/23
to WildFly
From what I can tell, multicast messages sent by devqtradevm01 are not being received by devqtradevm02 (and visa versa).
I suggest enabling diagnostics on your protocol stack [1] and use JGroups probe [2] to diagnose your networking issues.
[1] /subsystem=jgroups/stack=udp/transport=UDP:write-attribute(name=diagnostics-socket-binding, value=...)
Message has been deleted
vijen reddy
Apr 24, 2023, 5:03:01 PM4/24/23
to WildFly
Hi Paul,
[1] /subsystem=jgroups/stack=udp/transport=UDP:write-attribute(name=diagnostics-socket-binding, value=...)
ajp jgroups-tcp-fd modcluster
http jgroups-udp txn-recovery-environment
https jgroups-udp-fd txn-status-manager
jgroups-mping management-http
jgroups-tcp management-https
Can you please provide complete CLI command?? as I don't know what to mention as value in the above CLI command.
I found these and don't which one I should use to enable diagnostics?
[1] /subsystem=jgroups/stack=udp/transport=UDP:write-attribute(name=diagnostics-socket-binding, value=...)
ajp jgroups-tcp-fd modcluster
http jgroups-udp txn-recovery-environment
https jgroups-udp-fd txn-status-manager
jgroups-mping management-http
jgroups-tcp management-https
And also, honestly I don't understand how to use probe to diagnose networking issues though I have gone thru the above link :(
can you please give an example???
Paul Ferraro
Apr 25, 2023, 7:07:45 AM4/25/23
to WildFly
Can you also paste your <socket-binding-group/> and <interfaces/> sections from standalone xml?
Unless you've modified these, I suspect they are still listening on the loopback interface.
On Monday, April 24, 2023 at 5:03:01 PM UTC-4 vijen reddy wrote:
Hi Paul,
Can you please provide complete CLI command?? as I don't know what to mention as value in the above CLI command.I found these and don't which one I should use to enable diagnostics?
[1] /subsystem=jgroups/stack=udp/transport=UDP:write-attribute(name=diagnostics-socket-binding, value=...)
ajp jgroups-tcp-fd modcluster
http jgroups-udp txn-recovery-environment
https jgroups-udp-fd txn-status-manager
jgroups-mping management-http
jgroups-tcp management-https
You first need to create a socket-binding with an appropriate address/port for receiving multicast messages.
Then assign the diagnostics-socket-binding attribute to the name of the new socket-binding.
And also, honestly I don't understand how to use probe to diagnose networking issues though I have gone thru the above link :(can you please give an example???
Reply all
Reply to author
Forward
0 new messages