Serious bugs that never get fixed - recap
Prefontim
at getting resolution as i don't post anymore unless i have a test
case. However, all of the most serious bugs I face on a daily basis i
can't get resolution for because i can't come up with a test case.
With most of issues, i am debugging my fairly complicated webapps and
then things just stop working. It is impossible to know exactly what
caused it, so a test case is hard to make.
For a complicated app, my companies apps all have many iframes. Most
iframes have dojo in them, and dozens of other scripts. There are
many dom transformations and xhr requests happening all the time.
WHen things go wrong with these kinds of apps, it is hard to simulate
issues with simple test cases. It's hard just to reproduce them, in
general, as i don't noticed things are broken until i need them.
So, i thought i'd list all these bugs that i encounter on a daily
basis. Most of them have been happening to me since I started using
Firebug (1.2 or 1.3). What i am hoping is that maybe the Firebug
developers could take the opposite approach with these bugs. Instead
of requiring a simple test case, maybe they could test against a
complicated app until they can reproduce some of them. If not, maybe
others will post to this with experiences with these bugs.
1. 'console is not defined' - this will start happening at random
times. browser must be restarted
2. 'net panel stops working' - net panel will stop showing any
requests - browser must be restarted
3. console stops working - console stops showing any logging
4. script panel is empty - script panel will not have a script list -
browser must be restarted
5. clicking line number does not add breakpoint - you click on a line
number and it does not look like it added a breakpoint, even though it
does. the breakpoint shows up only after you click the Watch tab
6. lockup - browser locks up and gets unresponse script errors inside
firebugs code. must do force shutdown on browser.
7. console shows errors from non-activated pages in other tabs. - for
example, i might have Hotmail opened in a different tab, and i will
any errors from it in my tab.
John J Barton
On Jul 28, 10:21 am, Prefontim <Prefon...@hotmail.com> wrote:
> I have posted over 10 tickets to the Bugzilla system with much success
> at getting resolution as i don't post anymore unless i have a test
> case. However, all of the most serious bugs I face on a daily basis i
> can't get resolution for because i can't come up with a test case.
> With most of issues, i am debugging my fairly complicated webapps and
> then things just stop working. It is impossible to know exactly what
> caused it, so a test case is hard to make.
>
> For a complicated app, my companies apps all have many iframes. Most
> iframes have dojo in them, and dozens of other scripts. There are
> many dom transformations and xhr requests happening all the time.
> WHen things go wrong with these kinds of apps, it is hard to simulate
> issues with simple test cases. It's hard just to reproduce them, in
> general, as i don't noticed things are broken until i need them.
>
> So, i thought i'd list all these bugs that i encounter on a daily
> basis. Most of them have been happening to me since I started using
> Firebug (1.2 or 1.3). What i am hoping is that maybe the Firebug
> developers could take the opposite approach with these bugs. Instead
> of requiring a simple test case, maybe they could test against a
> complicated app until they can reproduce some of them. If not, maybe
> others will post to this with experiences with these bugs.
I'll give you the opposite story of frustration from my side: dojo
users complain about firebug problems but rarely report bugs with test
cases. There are 4 bug reports related to dojo, none with test cases:
http://code.google.com/p/fbug/issues/list?can=2&q=dojo
>
> 1. 'console is not defined' - this will start happening at random
> times. browser must be restarted
> 2. 'net panel stops working' - net panel will stop showing any
> requests - browser must be restarted
> 3. console stops working - console stops showing any logging
> 4. script panel is empty - script panel will not have a script list -
> browser must be restarted
> 5. clicking line number does not add breakpoint - you click on a line
> number and it does not look like it added a breakpoint, even though it
> does. the breakpoint shows up only after you click the Watch tab
> 6. lockup - browser locks up and gets unresponse script errors inside
> firebugs code. must do force shutdown on browser.
> 7. console shows errors from non-activated pages in other tabs. - for
> example, i might have Hotmail opened in a different tab, and i will
> any errors from it in my tab.
for this problem, but it's all hacks because Firefox does not support
this feature directly.
jjb
Prefontim
web desktop, and contains dozens of apps inside it. There are public
versions of it available, which could be used for debugging. However,
most of the bugs i mentioned happen while i am changing code, which
can't as easily be done on a public site.
I have searched Bugzilla and came up some ticktets that may be related
to my issues:
1. 2028
3. 2948
5. 2647
Anways, to follow up on my previous comments, my feeling is that the
issues i face is probably common to anyone who uses a complicated app,
because i have them so easliy. If the Firebug developers had some
kind of 'master complicated app', (e.g. - similar to google mail)
simple test cases would become obsolete as this app could be used to
duplicate all of the above issues.
John J Barton
On Jul 28, 11:20 am, Prefontim <Prefon...@hotmail.com> wrote:
> It's our own company's app, call webNetwork. It's basically a virtual
> web desktop, and contains dozens of apps inside it. There are public
> versions of it available, which could be used for debugging. However,
> most of the bugs i mentioned happen while i am changing code, which
> can't as easily be done on a public site.
>
> I have searched Bugzilla and came up some ticktets that may be related
> to my issues:
>
> 1. 2028
> 3. 2948
> 5. 2647
so how can I fix it?
>
> Anways, to follow up on my previous comments, my feeling is that the
> issues i face is probably common to anyone who uses a complicated app,
> because i have them so easliy. If the Firebug developers had some
> kind of 'master complicated app', (e.g. - similar to google mail)
> simple test cases would become obsolete as this app could be used to
> duplicate all of the above issues.
issues. If you were to provide such an application that duplicated the
above issues, then we could fix them.
One approach that I have used successfully in the past is for users to
run with Firebug tracing and post the traces to the issues list.
jjb
Prefontim
reliably duplicate any of these issues, i'll see about putting up a
public site for it.
Prefontim
reliably duplicate any of these issues, i'll see about putting up a
public site for it.
Bob Hassinger
Sent from my iPhone
On Jul 28, 2010, at 2:20 PM, Prefontim <Pref...@hotmail.com> wrote:
> It's our own company's app, call webNetwork. It's basically a virtual
> web desktop, and contains dozens of apps inside it. There are public
> versions of it available, which could be used for debugging. However,
> most of the bugs i mentioned happen while i am changing code, which
> can't as easily be done on a public site.
> ...
Between a person who sees a problem and a person who does not, who is better positioned to document it? The person who sees it has a context, which while perhaps large, is a great deal more specific than the universe if possibilities the FB developers would have to explore.
An observation on this general class of issues: perhaps it is inaccurate, but my sense in watching this forum, and others, is that quite frequently these difficult problems end up tracing back to something like an obscure error in the user's code or logic. Often a simple typo. The difficulty is basically in the way those often manifest themselves. Often is is very obscure, giving little clue to the true nature of what is going on.
My guess about concerns like the ones in this thread is that large, complex applications have a large number of places where code and logic errors can hide. It is not so much an issue of having a large application for the developers to test with as much as it is having one with particular errors and/or interactions between different programming aids packages.
Perhaps if somewhere in the browser and/or related elements like FB it could be possible to have some sort of an option to give more extensive information about what is going on inside. I suspect some of the most common issues around typos and the like have somewhat typical characteristics from the browser's point of view. Maybe more of that sort of thing could have options for being reported during development.
Example- buried in the typical browser there are points where syntax errors are compensated for. Most all the browsers tolerate a lot of erroneous code and do their best to move on rather then rejecting a page. Maybe it would be possible to have an option to expose those events. It would give developers a better chance to discover the source of problems (and maybe also help build high quality code in general).
ColinFine
On Jul 28, 7:20 pm, Prefontim <Prefon...@hotmail.com> wrote:
If the Firebug developers had some
> kind of 'master complicated app', (e.g. - similar to google mail)
> simple test cases would become obsolete as this app could be used to
> duplicate all of the above issues.
will continue to be disappointed.
While it is indeed helpful to have some complicated tests as well as
simple ones, they serve only the limited purpose of exercising the
system and showing that it is generally robust (or not :-).
In the first place there is not just one dimension of 'complicated':
Your complicated app is likely to be utterly different from the next
person's complicated app, and will in general show different problems.
Secondly, if one does show a problem, this may be helpful in showing
that the problem occurs, but be little help in isolating it, or
sometimes even in repoducing it. As Bob Hassinger says, usually people
who understand what the complicated system is trying to do will be
better at 1) knowing what they did to make it happen, 2) what kind of
thing they might do that is like that, and 3) what is likely to be
irrelevant. (They won't necessarily know these, of course, but they
can usually do them at least as well as the people who are maintaining
a general tool).
Thirdly, such tests typically take a lot of time to run, and may be
difficult to automate. This is a problem even for expensive commercial
tools where there is a budget for machine and people time to run them.
For volunteer projects it is extremely difficult.
Small, well-specified and controlled automatic tests are FAR more
effective than big amorphous ones (though again I say, there is a
place for the big ones).
Prefontim
complicated apps are not as good for automation or for tracking down
issues once they are found. However, they are much better than simple
tests at actually discovering issues in the first place. If a
complicated app was constructed by the Firebug team, and they simply
just used the app for 15 minutes, they would probably discover
multiple issues. Since they made the app, they would be in a position
to debug it effectively. Also, i would like to make a counter-point
to the following:
"Between a person who sees a problem and a person who does not, who
is better positioned to document it?"
problems, instead of us, and will therefore be in a much better
position to document it than we, since not only do they know the app,
but they also know FB, and where it might be failing. When you go to
test the app, you don't attack it by trying to duplicate or find a
specific issue. Instead, you just use the app and be aware of any
issues that arise. I say all this from experience, as I am also a
software developer. Whenever i take it upon myself to just 'use' a
complicated app, such as a user might, then I invariably find bugs.
Similarly, if a FB developer were to simply 'use' a complicated app,
they would also experience problems, but would then be in a better
position to fix them, since the whole duplication and simple test case
steps would not be unnecessary.
In conclusion, I am not suggesting that a complicated app testing
procedure be a substitute for your current processes, but only a
supplement to them.
Richard Quadling
The questions I would like to be asked at that stage would be...
1 - What happens?
2 - What do you do to make it happen?
If the mechanic is unsure, then maybe also ...
3 - What do you expect to happen?
Now, some of these may be answerable by me. Some, through experience,
may be answerable by the mechanic.
If I don't answer any of the questions, the mechanic is going to be in
a bit of a mess trying to find the fault before he can diagnose it and
repair it.
And, after all of that, whether or not the car is fixed, I'm going to
get a bill.
And if the mechanic is really good at his job (though not perfect), he
may be really popular.
And experience will tell you that those that cooperate with the
mechanic, answer his questions calmly and politely, will almost
certainly get a better response that those that don't. Those that
don't will either not get a repaired car in any reasonable time or i
Nearly all open source software is developed along community lines -
we all pull together. The vast majority of open source software is
developed voluntarily. If you had to pay for FireBug, how much would
you pay? Would you then expect a support contract?
I used to have "Standing on the shoulders of some very clever giants."
in my signature. I think I'll probably change it to "Standing on the
shoulders of some very clever and generous giants.".
The request for a test-case isn't a way to stop the problem being
fixed. It is part of the solution. But, unfortunately, it is a form of
triage. Many problems are reported which are quite plainly and
obviously due to the developer having NOT RTFM'd.
Some problems are simply due to the developer being too "clever" for
their own good and ending up tying themselves in knots and wanting a
hand and then blaming the first tool that comes to hand.
Many reasons.
But.
In all instances, if you can PROVE the problem, repeatedly and
reliably, then others can certainly help you.
John J Barton
On Jul 29, 5:39 am, Prefontim <Prefon...@hotmail.com> wrote:
> I would agree with the general tenor of these responses, in that
> complicated apps are not as good for automation or for tracking down
> issues once they are found. However, they are much better than simple
> tests at actually discovering issues in the first place. If a
> complicated app was constructed by the Firebug team, and they simply
> just used the app for 15 minutes, they would probably discover
> multiple issues. Since they made the app, they would be in a position
> to debug it effectively.
application using a wide spectrum of Web and XUL technologies. Rather
than 15 minutes, we use it all day every day. And yes it has allowed
us to discover many issues in Firebug code. (The application is called
"firebug" in case you are interested ;-).
I think that your analysis is missing three important aspects related
to the scale of these issues.
First, while you imagine that 'everyone' has the problems you have, it
just cannot be true. With 2.6 million users, only a tiny fraction
report problems. Now of course many people will not report problems,
but it has to be the case that most of these users do not see "serious
bugs". A more likely explanation is that you are special: you have a
more complex application, you use more sophisticated features, and you
use them in unusual ways.
Second, the spectrum of technologies that Firebug supports is large,
under-documented, and continually changing. A comprehensive test suite
for how these features even work does not exist, let alone a test
suite for how to debug these features. We know there are large gaps in
our test suite, but if we filled those gaps we would have no time to
update Firebug so there would be no point in having the test suite!
Third, we have one full time developer (Honza), one loaner (me), and a
good number of great contributors who make the difference between
floating and sinking. We just do not have the resources to solve
every problem.
Given this reality, the only way I can see for your bugs to be fixed
is for you to produce the test cases. Learn FBTrace (Firebug >
Firebug Icon Menu > Open Tracing) or even Chromebug (http://
getfirebug.com/wiki/index.php/Chromebug_User_Guide) if these bugs are
important to you. We are ready to help you with this part.
jjb
Sebo
As programmer you should know, that sometimes it's hard to find an
issue. And it's impossible, if you can't reproduce the problem
somebody is having. That's why the reporter and the developer have to
work together to solve the problem.
Unfortunately I am often seeing issues in the FB issues list, where
the developer is requesting for some more info, but the reporter is
not responding anymore. E. g. issue 2028 is such a case. The reporter
posted the issue, John needed more information, but just somebody else
is answering saying he can't reproduce the problem. Luckily I could,
so I created a simple test case for it with additional information.
Though this is not the worst example, since the reporter already tried
to give a test case with instructions to reproduce the problem. Other
issues are just like "Firebug is not running for me. Fix it!" (this is
what Richard described with the car example).
So the main thing here, is that as reporter of an issue you should be
at least responsive to the upcoming questions and cooperative enough
to help the developer get the bug fixed. Just a few examples where
this didn't work: issue 1941, 2216 and 2363.
In the cases you really can't track down an issue to a small
reproducable test case the Firebug Tracing Console might sometimes be
helpful. The trace log at least could give the developer a hint where
the problem lies even when he can't reproduce it by himself e. g. due
to another system configuration.
John let me join the team, because I created test cases for issues,
the reporter didn't provide. Though it's hard to do so, because the
developer has to spend a big part of his time trying to understand
what a bug reporter might have thought and based on that having to
figure out by himself how this problem is reproducable.
We all want Firebug to improve. So let's work together to reach this
goal!
Sebastian
Garrett Smith
>
>
> On Jul 29, 5:39 am, Prefontim <Prefon...@hotmail.com> wrote:
>> I would agree with the general tenor of these responses, in that
>> complicated apps are not as good for automation or for tracking down
>> issues once they are found. However, they are much better than simple
>> tests at actually discovering issues in the first place. If a
>> complicated app was constructed by the Firebug team, and they simply
>> just used the app for 15 minutes, they would probably discover
>> multiple issues. Since they made the app, they would be in a position
>> to debug it effectively.
>
> We already do this. We use the Firebug code to debug a complex 40kloc
> application using a wide spectrum of Web and XUL technologies. Rather
> than 15 minutes, we use it all day every day. And yes it has allowed
> us to discover many issues in Firebug code. (The application is called
> "firebug" in case you are interested ;-).
>
> I think that your analysis is missing three important aspects related
> to the scale of these issues.
>
> First, while you imagine that 'everyone' has the problems you have, it
> just cannot be true. With 2.6 million users, only a tiny fraction
> report problems.
OK, but don't expect a lack of reports for a lack of errors.
Sometimes it is necessary to restart Firefox to get Firebug working.
SOmetimes just opening a new tab works and that is easier, so I
usually try that first. I have experienced all of the OP's 7 items, so
that's how I know.
I'm sorry I can't be of more help. I don't know what triggers the
problems and so I haven't been able to file a proper bug report.
Maybe the someone has time can take one bug and investigate it and
hopefully provide more detail (steps to repro, etc).
Garrett
John J Barton
On Jul 30, 10:00 am, Garrett Smith <dhtmlkitc...@gmail.com> wrote:
1. Open firebug tracing panel with option ERRORS
2. The first time you have a problem, check the trace.
3. Whether or not there is any info in the trace, describe exactly
what the problem is. 'net panel stops working' is not enough. Is it
blank? is the Console panel still showing XHR? If you set NET on the
tracing options do you see stuff?
>
> Maybe the someone has time can take one bug and investigate it and
> hopefully provide more detail (steps to repro, etc).
isn't any other "someone" able to see your problems. No one from the
Firebug team can help you, we do not see these problems.
jjb
>
> Garrett
Garrett Smith
No, not only us two; the OP is bringing up bugs that others have
reported. Looking at the bugtracker:
2948: Console stops working (No XHR or log outputs)
With comment from " lisman.aaron, Apr 12, 2010" and " gullevek, Jun
17, 2010" confirming the same.
I have other issues at the moment and a ton of stuff on my plate.
Sorry, there are real issues. They don't bug me enough to motivate me
to investigate further. I'm just busy with a bunch of stuff. Thanks.
Prefontim
test case or trace. However, it can still be a good idea to post
issues without those things, in the hopes that they or others will
have steps to duplicate.
But, regardless, the point of this post was not to say that any bug
ticket should get fixed, even ones without test cases. Instead, it
was to suggest an alternate way of generating bug tickets. One which
would be a supplement to the issue-finding process, and could be a
good way to find issues that affect complicated apps, like mine. But,
i do very much appreciate all the contributors efforts, and would not
want to place any extra burden on them, unless they chose it that way.
The best counter-suggestion was to use the trace logs, which I
subsequently did in the following ticket, to much success:
http://code.google.com/p/fbug/issues/detail?id=3292
So, i will continue to use trace logs to figure out the remainder of
my complicated issues.
On Jul 31, 2:42 am, Garrett Smith <dhtmlkitc...@gmail.com> wrote: