Advice on where to start with garbled/junk page output for some requests
112 views
Skip to first unread message
AWDSOFT
Nov 2, 2015, 3:27:04 PM11/2/15
to Lucee
Hi everyone,
I've been working on converting an application from ACF9 to Lucee 4 and there is something very odd going on. Our testers are reporting that every so often they receive page output that is just a bunch of mangled/garbage characters. I have seen it myself once or twice (out of hundreds of requests) so I know it's not a false report; just rare. Reloading the page will result in the correct output. I'm going to be collecting more information as I can but any direction or ideas on where to even begin would be very much appreciated!
A little info about our environment:
- Lucee 4.5.1.024 / Apache Tomcat/8.0.24 / 1.8.0_45 (Oracle Corporation) 64bit
- Windows Server 2008 R2
- Using multi-instance installation following modified instruction from here: http://utdream.org/assets/content/files/railo_multi-instance-howto.pdf
- Basically running full installer with custom ports and manual config of the BonCode IIS connector.
Jordan Michaels
Nov 2, 2015, 3:42:29 PM11/2/15
to lu...@googlegroups.com
More information about the 'garbled characters' would be helpful, however, I've had experiences recently that could be similar. In my own recent experiences, I had a web application firewall that was filtering out some custom font files. This caused custom icons to change to "garbled characters" instead of the font icons that they were supposed to be. Do you use custom font icons in your application? Are you certain these custom fonts (the actual font files and their related CSS) are coming through properly and being referenced properly? If not, this can lead to 'garbled characters'.
Cross-site scripting protections (CORS) can also be a cause of this issue. Try viewing your site through an external proxy to help identify issues like these.
Hope this helps!
Kind regards,
Jordan Michaels
----- Original Message -----
From: "AWDSOFT" <awd...@gmail.com>
To: "Lucee" <lu...@googlegroups.com>
Sent: Monday, November 2, 2015 12:27:03 PM
Subject: [Lucee] Advice on where to start with garbled/junk page output for some requests
Hi everyone,
I've been working on converting an application from ACF9 to Lucee 4 and
there is something very odd going on. Our testers are reporting that every
so often they receive page output that is just a bunch of mangled/garbage
characters. I have seen it myself once or twice (out of hundreds of
requests) so I know it's not a false report; just rare. Reloading the page
will result in the correct output. I'm going to be collecting more
information as I can but any direction or ideas on where to even begin
would be very much appreciated!
A little info about our environment:
- Lucee 4.5.1.024 / Apache Tomcat/8.0.24 / 1.8.0_45 (Oracle Corporation)
64bit
- Windows Server 2008 R2
- Using multi-instance installation following modified instruction from
here: http://utdream.org/assets/content/files/railo_multi-instance-howto.pdf
- Basically running full installer with custom ports and manual
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/b554f5b5-c03e-4d85-a582-7f9f4d2a5b26%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Cross-site scripting protections (CORS) can also be a cause of this issue. Try viewing your site through an external proxy to help identify issues like these.
Hope this helps!
Kind regards,
Jordan Michaels
----- Original Message -----
From: "AWDSOFT" <awd...@gmail.com>
To: "Lucee" <lu...@googlegroups.com>
Sent: Monday, November 2, 2015 12:27:03 PM
Subject: [Lucee] Advice on where to start with garbled/junk page output for some requests
Hi everyone,
I've been working on converting an application from ACF9 to Lucee 4 and
there is something very odd going on. Our testers are reporting that every
so often they receive page output that is just a bunch of mangled/garbage
characters. I have seen it myself once or twice (out of hundreds of
requests) so I know it's not a false report; just rare. Reloading the page
will result in the correct output. I'm going to be collecting more
information as I can but any direction or ideas on where to even begin
would be very much appreciated!
A little info about our environment:
64bit
- Windows Server 2008 R2
- Using multi-instance installation following modified instruction from
here: http://utdream.org/assets/content/files/railo_multi-instance-howto.pdf
- Basically running full installer with custom ports and manual
config of the BonCode IIS connector.
--
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/b554f5b5-c03e-4d85-a582-7f9f4d2a5b26%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
AWDSOFT
Nov 2, 2015, 4:57:49 PM11/2/15
to Lucee
Hi Jordan,
Thanks for your reply!
To answer your question; no, we aren't using any custom fonts in this application. Let me try to elaborate on some details about the request (hopefully I can actually post an example of the output soon).
- I mean to say that the "output" of the page is garbled/mangled. The characters appearing in the output are plainly text but have no meaning. Think something like "Q9%h!2JJ~0#cRp&" but much longer.
- There is absolutely no rendered output by the browser (no icons, text, tables, etc) other than the large string of random characters.
- The request takes an unusually long amount of time to load.
- I do not have the status code returned to the browser yet.
- Nothing appears in the exception.log.
Michael Sprague
Nov 2, 2015, 5:06:04 PM11/2/15
to lucee
If the data for the garbled pages is coming from a database it could be an issue with the character encoding settings. I've seen similar issues when the data in the database is stored with a different charset than the original source of the data. Hope that helps.
Mike
--
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/6a13bfb5-b732-4a75-bb82-3c0c0771ea22%40googlegroups.com.
AWDSOFT
Nov 2, 2015, 7:03:12 PM11/2/15
to Lucee
Thanks for the reply Mike!
In this case, I don't think that would be the issue since the page is a "standard" CFML page with a mixture of HTML elements and CFML markup. Also, reloading the page will show the correct content which I would expect not to be the case if it were a database content encoding issue. Good idea though!
Gert Franz
Nov 2, 2015, 7:20:41 PM11/2/15
to lu...@googlegroups.com
Could it be that those pages are encrypted in acf?.
Gert
Sent from somewhere on the road
Sent from somewhere on the road
To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/5d50371c-28b4-4f5d-a23c-df312633a849%40googlegroups.com.
Hugo Ahlenius
Nov 3, 2015, 2:17:41 AM11/3/15
to lu...@googlegroups.com
Hi - what if what you see is binary stuff, that the gzip compression is not working as it should... ?
AWDSOFT wrote on 2015-11-02:
> Hi Jordan,
>
> Thanks for your reply!
>
> To answer your question; no, we aren't using any custom fonts in this
> application. Let me try to elaborate on some details about the request
> (hopefully I can actually post an example of the output soon).
>
> * I mean to say that the "output" of the page is garbled/mangled.
> * I do not have the status code returned to the browser yet.
> * Nothing appears in the exception.log.
> Corporation) 64bit * Windows Server 2008 R2 * Using multi-instance
> * Basically running full installer with custom ports and manual config
AWDSOFT wrote on 2015-11-02:
> Hi Jordan,
>
> Thanks for your reply!
>
> To answer your question; no, we aren't using any custom fonts in this
> application. Let me try to elaborate on some details about the request
> (hopefully I can actually post an example of the output soon).
>
> The characters appearing in the output are plainly text but have no
> meaning. Think something like "Q9%h!2JJ~0#cRp&" but much longer.
> * There is absolutely no rendered output by the browser (no icons,
> meaning. Think something like "Q9%h!2JJ~0#cRp&" but much longer.
> text, tables, etc) other than the large string of random characters.
> * The request takes an unusually long amount of time to load.
> * I do not have the status code returned to the browser yet.
> * Nothing appears in the exception.log.
>
>
> On Monday, November 2, 2015 at 1:27:04 PM UTC-7, AWDSOFT wrote:
>
> Hi everyone,
>
> I've been working on converting an application from ACF9 to
> Lucee 4 and there is something very odd going on. Our testers are
> reporting that every so often they receive page output that is just a
> bunch of mangled/garbage characters. I have seen it myself once or twice
> (out of hundreds of requests) so I know it's not a false report; just
> rare. Reloading the page will result in the correct output. I'm going to
> be collecting more information as I can but any direction or ideas on
> where to even begin would be very much appreciated!
>
> A little info about our environment:
>
> * Lucee 4.5.1.024 / Apache Tomcat/8.0.24 / 1.8.0_45 (Oracle
>
> On Monday, November 2, 2015 at 1:27:04 PM UTC-7, AWDSOFT wrote:
>
> Hi everyone,
>
> I've been working on converting an application from ACF9 to
> Lucee 4 and there is something very odd going on. Our testers are
> reporting that every so often they receive page output that is just a
> bunch of mangled/garbage characters. I have seen it myself once or twice
> (out of hundreds of requests) so I know it's not a false report; just
> rare. Reloading the page will result in the correct output. I'm going to
> be collecting more information as I can but any direction or ideas on
> where to even begin would be very much appreciated!
>
> A little info about our environment:
>
> Corporation) 64bit * Windows Server 2008 R2 * Using multi-instance
> installation following modified instruction from here:
> http://utdream.org/assets/content/files/railo_multi-instance-howto.pdf
> <http://utdream.org/assets/content/files/railo_multi-instance-howto.pdf>
> http://utdream.org/assets/content/files/railo_multi-instance-howto.pdf
> * Basically running full installer with custom ports and manual config
Nando Breiter
Nov 3, 2015, 7:58:52 AM11/3/15
to lu...@googlegroups.com
Aliens intercepting the web request trying to communicate with you (or your users)? ;-)
Aria Media Sagl
Via Rompada 40
6987 Caslano
Switzerland
+41 (0)91 600 9601
+41 (0)76 303 4477 cell
skype: ariamedia
--
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/017001d11607%249512e280%24bf38a780%24%40oxel.net.
AWDSOFT
Nov 3, 2015, 12:11:41 PM11/3/15
to Lucee
Hi Gert,
Thanks for your reply!
No, these pages are not ACF encrypted files. They aren't special in any way; just plain text ".cfm" files on the local file system which can be opened in Notepad, etc.
AWDSOFT
Nov 3, 2015, 12:14:01 PM11/3/15
to Lucee
Hi Hugo,
That's an interesting idea... I noticed in the Lucee admin there is a setting for GZIP compression which I thought was interesting since I'm sure our IIS is already setup to use GZIP. How does that work if both are set to use GZIP? Maybe my issue is "double-zip" (but only rarely)?
Hugo Ahlenius
Nov 4, 2015, 4:41:04 AM11/4/15
to lu...@googlegroups.com
AWDSOFT wrote on 2015-11-03:
> That's an interesting idea... I noticed in the Lucee admin there is a
> setting for GZIP compression which I thought was interesting since I'm
> sure our IIS is already setup to use GZIP. How does that work if both
> are set to use GZIP? Maybe my issue is "double-zip" (but only rarely)?
If this is gzip-related: More likely is that the headers get bungled or the connection gets interrupted.
> That's an interesting idea... I noticed in the Lucee admin there is a
> setting for GZIP compression which I thought was interesting since I'm
> sure our IIS is already setup to use GZIP. How does that work if both
> are set to use GZIP? Maybe my issue is "double-zip" (but only rarely)?
AWDSOFT
Nov 18, 2015, 12:37:41 PM11/18/15
to Lucee
Update time! This continues to happen rarely but I did finally get a sample of the output and a little more information. Since the original post we have updated to 4.5.2.018. Here is an example of the output by the browser (there are hundreds of lines, this is just some of the first):
‹ ½YmSÛH þ ýŠA—ZÙ‰,Y† 0˜”±Í†Tx)0›»¢R[ƒ4¶ dÉ 0.–ÿ¾Ý3’-Ù†lî’ • M÷ôû<Ý£ìotÏ:ýÿœ÷ÈHŽCr~uøù¸CÌšë~Ùì¸n·ß%ÿþØ?ùL<§Nú‚F —<Žh躽S“˜#)'M× N§ÎtÓ‰ÅÐí_¸ (ËÃÍÙcM v: Ì c_)| ‡
This time, I also received an error for the the request in our logs. Here is the stack trace:
java.lang.ThreadDeath at java.lang.Thread.stop(Thread.java:850):850 at
lucee.commons.io.StopThread.run(SystemUtil.java:1091):1091 at
lucee.runtime.exp.NativeException.<init>(NativeException.java:51):51 at
lucee.runtime.op.Caster.toPageException(Caster.java:3046):3046 at
lucee.runtime.type.UDFImpl._call(UDFImpl.java:338):338 at
lucee.runtime.type.UDFImpl.call(UDFImpl.java:229):229 at
lucee.runtime.ComponentImpl._call(ComponentImpl.java:642):642 at
lucee.runtime.ComponentImpl._call(ComponentImpl.java:524):524 at
lucee.runtime.ComponentImpl.call(ComponentImpl.java:1761):1761 at
lucee.runtime.util.VariableUtilImpl.callFunctionWithoutNamedValues(VariableUtilImpl.java:742):742 at
lucee.runtime.PageContextImpl.getFunction(PageContextImpl.java:1590):1590 at
{subFolder}.{myFile}_cfm$cf.call(D:\Webs\{myFile}.cfm:495):495 at
lucee.runtime.PageContextImpl.doInclude(PageContextImpl.java:951):951 at lucee.runtime.PageContextImpl.doInclude(PageContextImpl.java:903):903 at
lucee.runtime.listener.ModernAppListener._onRequest(ModernAppListener.java:223):223 at
lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:35):35 at
lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2262):2262 at
lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2225):2225 at
lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:456):456 at
lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:47):47 at
javax.servlet.http.HttpServlet.service(HttpServlet.java:729):729 at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291):291 at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206):206 at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52):52 at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239):239 at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206):206 at
sun.reflect.GeneratedMethodAccessor200.invoke(Unknown Source):-1 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43):43 at
java.lang.reflect.Method.invoke(Method.java:497):497 at
com.intergral.fusionreactor.j2ee.filterchain.WrappedFilterChain.doFilter(WrappedFilterChain.java:97):97 at
com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.doNext(FusionReactorRequestHandler.java:472):472 at
com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.doHttpServletRequest(FusionReactorRequestHandler.java:312):312 at
com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.doFusionRequest(FusionReactorRequestHandler.java:192):192 at
com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.handle(FusionReactorRequestHandler.java:507):507 at
com.intergral.fusionreactor.j2ee.filter.FusionReactorCoreFilter.doFilter(FusionReactorCoreFilter.java:36):36 at
sun.reflect.GeneratedMethodAccessor199.invoke(Unknown Source):-1 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43):43 at
java.lang.reflect.Method.invoke(Method.java:497):497 at
com.intergral.fusionreactor.j2ee.filterchain.WrappedFilterChain.doFilter(WrappedFilterChain.java:79):79 at
sun.reflect.GeneratedMethodAccessor198.invoke(Unknown Source):-1 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43):43 at
java.lang.reflect.Method.invoke(Method.java:497):497 at
com.intergral.fusionreactor.agent.filter.FusionReactorStaticFilter.doFilter(FusionReactorStaticFilter.java:53):53 at
com.intergral.fusionreactor.agent.pointcuts.NewFilterChainPointCut$1.invoke(NewFilterChainPointCut.java:41):41 at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java):-1 at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219):219 at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106):106 at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502):502 at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142):142 at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79):79 at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88):88 at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518):518 at
org.apache.coyote.ajp.AbstractAjpProcessor.process(AbstractAjpProcessor.java:844):844 at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:668):668 at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1527):1527 at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1484):1484 at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142):1142 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617):617 at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61):61 at
java.lang.Thread.run(Thread.java:745):745On Monday, November 2, 2015 at 1:27:04 PM UTC-7, AWDSOFT wrote:
Paul Klinkenberg
Nov 19, 2015, 4:44:43 AM11/19/15
to lu...@googlegroups.com
Hi there,
I see in the ST you are using Fusionreactor. Good choice :)
What does fusionreactor say about these requests?
Also, can you post the line D:\Webs\{myFile}.cfm:495 and surrounding code?
The junk characters are probably partial binary data. What was the page in question supposed to output? a pdf file? Image?
Also, you might want to check the Tomcat error logs; they might give you an idea to what's going on.
Kind regards,
Paul Klinkenberg
--
Love Lucee? Become a supporter and be part of the Lucee project today! - http://lucee.org/supporters/become-a-supporter.html
---
Love Lucee? Become a supporter and be part of the Lucee project today! - http://lucee.org/supporters/become-a-supporter.html
---
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/35138272-252b-49f3-aa1c-23177f86a040%40googlegroups.com.
Paul Klinkenberg
Nov 19, 2015, 5:07:19 AM11/19/15
to lu...@googlegroups.com
... I did not read the full email thread before replying. Sorry 'bout that.
I see you wrote "The request takes an unusually long amount of time to load." That kind of indicates a timeout caused the problem. Please check the Fusionreactor settings for request timeout handling; it probably says "abort on timeout".
If that is the case, you could change it to "warn by email", to allow for debugging.
As to why it took so long, Fusionreactor is probably your friend again.
Kind regards,
Paul Klinkenberg
AWDSOFT
Nov 19, 2015, 12:55:29 PM11/19/15
to Lucee
Hi Paul,
Thanks for your reply!
I'm not sure I could live without FusionReactor; it's such a great tool for debugging/monitoring. Since this issue crops up so rarely I have not been able to view the request in FR as it happens (or even in the history since we commonly have 30+ requests per second). We have a separate error logging system and that's how I was able to provide the stack trace. I don't actually have FR setup to use "WebRequest Runtime Protection" so there shouldn't be any killing (by FR anyway) going on.
Good idea looking at the Tomcat logs for this most recent case; might turn up something interesting there.
The line in that file is using cfsavecontent to build a potentially large string of CSV data. It is frequently used though and does not consistently fail in this way.
Do you suppose that this issue could simply be timeouts that sometimes bomb out with binary dumps? Our error handling is supposed to display a nice error page but in these cases it's not happening.
...
Reply all
Reply to author
Forward
0 new messages