Gitblit: Unable to access UI file formats

[Anjan Tummalapalli]

Hi Team,

We are using Gerrit V2.11 and Gitblit vv2.11.10-2-g1498dff

We are able to access all the files in our repos but except some file formats related to UI components mentioned some of them below

js, css, scss, font formats (svg, ttf, eot), images (png, jpg) etc..

Here are some screenshots for reference

This change contains 2 file modifications (js and html)

Able to access diff of html file as below

But unable to access js file being in the same change as below

can someone please help me in getting out of this.. am i missing any config changes here?

Best Regards,

Anjan

[Luca Milanesio]

Hi Anjan,

can you show the HTTP calls made by the browser to highlight the JS full URL?

Do you see anything unusual related to that call in the Gerrit error_log?

Luca.

On 5 Oct 2016, at 09:01, Anjan Tummalapalli <anjan.tum...@gmail.com> wrote:

Hi Team,

We are using Gerrit V2.11 and Gitblit vv2.11.10-2-g1498dff

We are able to access all the files in our repos but except some file formats related to UI components mentioned some of them below

js, css, scss, font formats (svg, ttf, eot), images (png, jpg) etc..

Here are some screenshots for reference

<change_link.png>

This change contains 2 file modifications (js and html)

Able to access diff of html file as below

<working_link.png>

But unable to access js file being in the same change as below

<not_working_page.png>

can someone please help me in getting out of this.. am i missing any config changes here?

Best Regards,

Anjan

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Anjan Tummalapalli]

Thanks Luca for quick reply

Here are my observations

error_log:

saw below error very frequently with huge number of lines

[2016-10-05 06:57:06,291] ERROR org.apache.wicket.Session : Exception when detaching/serializing page
java.lang.StackOverflowError
        at java.io.ObjectOutputStream.writeHandle(ObjectOutputStream.java:1204)
        at java.io.ObjectOutputStream.writeClassDesc(ObjectOutputStream.java:1226)
        at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1426)
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
        at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1377)
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1173)
        at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
        at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
        at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
        at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1377)
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1173)
        at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
        at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
        at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
        at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1377)
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1173)
        at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
        at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
        at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
        at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1377)
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1173)

still continues..

httpd_log:

<ip address> - anjan [05/Oct/2016:06:49:08 -0400] "GET /plugins/reviewers/static/reviewers/reviewers.nocache.js HTTP/1.1" 200 7719 "http://gerrit.XXX.com:8080/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36"

<ip address> - anjan [05/Oct/2016:06:51:13 -0400] "GET /plugins/gitblit/blob/UI.git/f1535d55f1d6902b24149096c95fa133d9e79046/GSApp%2Fjs%2FXXX%2Futils%2FreportUtils%2Fprocessors%2FBasicChartProcessor.js HTTP/1.1" 404 9 "http://gerrit.XXX.com:8080/plugins/gitblit/commit/UI.git/f1535d55f1d6902b24149096c95fa133d9e79046" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36"

<ip address> - anjan [05/Oct/2016:06:46:27 -0400] "GET /plugins/gitblit/blob/UI.git/ecb69bfd1789b097f16554777b134a3fe131a2a6/Styles%2Fscss%2Fmodule%2Fscorecards%2F_scorecard2.scss HTTP/1.1" 404 9 "http://gerrit.XXX.com:8080/plugins/gitblit/commit/UI.git/ecb69bfd1789b097f16554777b134a3fe131a2a6" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36"

<ip address> - anjan [05/Oct/2016:06:49:10 -0400] "GET /plugins/gitblit/blob/UI.git/ecb69bfd1789b097f16554777b134a3fe131a2a6/Styles%2Fscss%2Fmodule%2Fscorecards%2F_scorecard2.scss HTTP/1.1" 404 9 - "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36"

Please let me know if you need more info..

Best Regards,

Anajn

On Wed, Oct 5, 2016 at 3:49 PM, Luca Milanesio <luca.mi...@gmail.com> wrote:

Hi Anjan,

can you show the HTTP calls made by the browser to highlight the JS full URL?

Do you see anything unusual related to that call in the Gerrit error_log?

Luca.

On 5 Oct 2016, at 09:01, Anjan Tummalapalli <anjan.tum...@gmail.com> wrote:

Hi Team,

We are using Gerrit V2.11 and Gitblit vv2.11.10-2-g1498dff

We are able to access all the files in our repos but except some file formats related to UI components mentioned some of them below

js, css, scss, font formats (svg, ttf, eot), images (png, jpg) etc..

Here are some screenshots for reference

<change_link.png>

This change contains 2 file modifications (js and html)

Able to access diff of html file as below


<working_link.png>

But unable to access js file being in the same change as below

<not_working_page.png>

can someone please help me in getting out of this.. am i missing any config changes here?

Best Regards,

Anjan

--
--
To unsubscribe, email repo-discuss+unsubscribe@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss+unsubscribe@googlegroups.com.

[lucamilanesio]

Looks like a recursive references in Java objects which causes a stack overflow.

Can you locate the bottom of the stack, where actually the serialisation calls start?

Luca.

--
--
To unsubscribe, email repo-discuss...@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.

[Anjan Tummalapalli]

Here is the bottom of that exception

and observed some other exceptions in error_log

[2016-10-05 10:12:16,641] WARN  com.google.gerrit.sshd.GerritServerSession : Exception caught
java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
        at sun.nio.ch.IOUtil.read(IOUtil.java:197)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
        at org.apache.mina.transport.socket.nio.NioProcessor.read(NioProcessor.java:302)
        at org.apache.mina.transport.socket.nio.NioProcessor.read(NioProcessor.java:45)
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:694)
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:668)
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:657)
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:67)
        at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1121)
        at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
[2016-10-05 10:14:30,757] WARN  com.google.gerrit.sshd.GerritServerSession : Exception caught
org.apache.sshd.common.SshException: Unsupported protocol version: quit
        at org.apache.sshd.server.session.ServerSession.readIdentification(ServerSession.java:165)
        at org.apache.sshd.common.session.AbstractSession.messageReceived(AbstractSession.java:302)
        at org.apache.sshd.common.AbstractSessionIoHandler.messageReceived(AbstractSessionIoHandler.java:54)
        at org.apache.sshd.common.io.mina.MinaService.messageReceived(MinaService.java:105)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:854)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:542)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:48)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:943)
        at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:109)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:542)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:535)
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:714)
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:668)
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:657)
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:67)
        at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1121)
        at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)

Best Regards,

Anjan

--
--
To unsubscribe, email repo-discuss+unsubscribe@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss+unsubscribe@googlegroups.com.

[Luca Milanesio]

... you're missing the end :-)

We need to find out which object GitBlit was trying to serialise ! I can't see Gitblit in the stacktrace anywhere.

Nothing related to the GitBlit plugin.

[Anjan Tummalapalli]

That is the end of that exception..

Is it working at your end?

Please let me know if you need more info

Best Regards,

Anjan

[Luca Milanesio]

Yep, works like a charm for me.

I have noticed though that the stack-traces comes actually from Wicket and not Gitblit:

[2016-10-05 06:57:06,291] ERROR org.apache.wicket.Session : Exception when detaching/serializing page
java.lang.StackOverflowError

The problem with Gitblit is that is (unfortunately) based on a well-known to be non-scalable framework ... Wicket :-(

What makes Wicket not scalable at all is the way that session is managed: serialisation of *ALL* Java objects on the wire at *each* request.

Here the problem is that Wicket is even blowing up for your session, possibly because GitBlit has stored some Java objects with recursive dependencies.

Gitblit is nice, but isn't perfect and more importantly isn't scalable :-(

Can you reproduce the same issue with Gitblit standalone?

(try to download Gitblit GO and point to the same Git directory)

Luca.

[tw20...@gmail.com]

On Thursday, October 6, 2016 at 9:07:52 AM UTC+2, lucamilanesio wrote:

Here the problem is that Wicket is even blowing up for your session, possibly because GitBlit has stored some Java objects with recursive dependencies.

Exactly. And the recursive data structure are JGit RevCommits, which unfortunately implement java.io.Serializable and contain parent links, and even an internal hash collision chain link. See https://github.com/gitblit/gitblit/pull/1141 .

[lucamilanesio]

I saw Tom's PR on Gitblit, as soon that it is merged you could then just rebuild the plugin and the problem will be solved :-)

Luca.

[Anjan Tummalapalli]

Thanks a lot Luca.. will do that for sure and update you with the status