WiFi Sync - java.net.SocketTimeoutException

[Dan]

Hi,

I cannot get WiFi sync to work. The initial connect with the PIN is ok and the first sync also. All tasks are synced. However if I try another sync after roughly a minute the error "java.net.SocketTimeoutException: Read timed out" pops up. This is then the case for every sync.

I have tried/analyzed dozen of things but with no success:

- Resetting

- Reducing amount of tasks

- Checked network settings (i.a. Windows firewall) -> However it does not really make any sense that it is network related as the first sync works or am I mistaken?!

- Checked the logs.

- Tried different MLO versions.

- Contacted MLO support ages ago but it ended with no solution.

- etc.

Has anybody got an idea? Thanks for any help!

Best regards,

Dan

[Dan]

To make it more concrete. This is seemingly the interesting part from the wifi log:

000005bc 11:06:36:                   ENTER: CWiFiServerProtocol::ProcessGetModificationsRequest()
000005bc 11:06:36:                     Calling callback...
000005bc 11:08:06:                     Callback returned.
000005bc 11:08:07:                     ENTER: CWiFiProtocol::SendCommandV()
000005bc 11:08:07:                       SendCommandV buffer - 30e64a0
000005bc 11:08:07:                       SendCommandV nLen 12
000005bc 11:08:07:                       SendString(`OK 62360,209
`)
000005bc 11:08:07:                       SendBlock(size: 13)
000005bc 11:08:07:                       13 bytes sent
000005bc 11:08:07:                     LEAVE: CWiFiProtocol::SendCommandV()
000005bc 11:08:07:                     SendBlock(size: 6000)
000005bc 11:08:07:                     6000 bytes sent
000005bc 11:08:07:                     SendBlock(size: 6000)
000005bc 11:08:07:                     6000 bytes sent
000005bc 11:08:07:                     SendBlock(size: 6000)
000005bc 11:08:07:                     6000 bytes sent
000005bc 11:08:07:                     SendBlock(size: 6000)
000005bc 11:08:07:                     6000 bytes sent
000005bc 11:08:07:                     SendBlock(size: 6000)
000005bc 11:08:07:                     6000 bytes sent
000005bc 11:08:07:                     SendBlock(size: 6000)
000005bc 11:08:07:                     6000 bytes sent
000005bc 11:08:07:                     SendBlock(size: 6000)
000005bc 11:08:07:                     6000 bytes sent
000005bc 11:08:07:                     SendBlock(size: 6000)
000005bc 11:08:07:                     6000 bytes sent
000005bc 11:08:07:                     SendBlock(size: 6000)
000005bc 11:08:07:                     6000 bytes sent
000005bc 11:08:07:                     SendBlock(size: 6000)
000005bc 11:08:07:                     6000 bytes sent
000005bc 11:08:07:                     SendBlock(size: 2360)
000005bc 11:08:07:                     2360 bytes sent
000005bc 11:08:07:                     Cleaning up
000005bc 11:08:07:                     Cleaned up
000005bc 11:08:07:                   LEAVE: CWiFiServerProtocol::ProcessGetModificationsRequest() = 1
000005bc 11:08:07:                   ENTER: CSocketClient::GetLine()
00002648 11:08:36:                     ERROR: Timeout obtaining socket read availability (code 0)

Why can the callback take 1,5 minutes? Is this regarding the PC or the Android device?

If I interpret the log correctly it then continues but my guess is that because of the lost 1,5 minutes the socket timeout is reached and therefore it is closed.

In general the communication over my wifi is fast between the devices and I haven't had any issues apart this one with MyLifeOrganized?!

I am simply stumped :-(

[Dan]

I've figured it out and fixed it.

------------------

For anyone who is interested. The sync process initially syncs all the delta data from Android to the PC. This worked flawlessly. Afterwards the delta data from the PC to the Android device is synced.
Here always the problem occurred. It had to do with the context and and flag elements in the app! These were duplicated. This was noticeable as the same names were existing multiple times but had added "1", "11" and "2" at the end.
I myself have never created or modified any contexts or flags. Seemingly they must have been duplicated during an older app update or after restoring the app. What was very noticeable was that the flags had 2 designs.
Some with one color (material design) and some with let's call it pixel art. The pixel art ones I strongly believe where the ones MyLifeOrganized used with an older version.
Seemingly the upgrade did not convert these flags correctly. This then caused the issue at hand.
But these are all my assumptions as I am not the developer of this product ;-)

The solution was simply to remove all of the duplicated entries. Afterwards the sync was fine again.

Just as a note to myself. These are the currently working versions so that I know what to go back to if it should break again in the future:
- Android = 3.4.2
- PC = 5.1.0

There is no way of downloading older versions via the homepage, or is there?

------------------

In general I am happy that I can now finally since 9.9.2018 (!) can use the sync again. So basically can start using the PC version again.
However getting to this point was an ordeal! I tried so many times to figure it out and contacted support and just gave up with frustration.

I don't want to come across as mister know-it-all. My goal is simply to give suggestions to make it easier to solve such issues in the future :-)
Your software has great and unique features! However if these aren't usable you might loose customers. The WiFi sync (I don't want cloud) between PC and Android was the reason I bought the software in the first place.
As seemingly nobody else has this on the market I stuck with it even though I had given up hope that this sync issue would ever be solved.

I totally get that every software with a certain complexity will have bugs. Also as I am a developer myself I fully understand that figuring out a customer issue that you cannot reproduce yourself can be the absolute worst.

However it would be helpful to have better logging. For one maybe for the customer but more importantly for you :-)
The exception and the log totally misleads you. One does believe it must have to do with the network. So all the wrong places are checked over and over: firewall, router, Android settings, network traffic etc.
Via procmon of Sysinternals Suite I could figure out that there is a debug registry setting that MLO checks. However setting it did not change the logging for me. Maybe I used the wrong value or something else is also needed.
Maybe that would be the right place to use in these cases.

Just as a suggestion. In cases where I could not figure out a customer problem I created a special debug version with extensive logging to track down how far the code comes until the issue occurs.
This is usually the last resort but then most of times very productive to find and fix it.

In general having to ask the customer for their data file to analyze the problem is not ideal. I for one have some private data in my tasks which I do not want to give to strangers.
So maybe a mechanism to override the title and notes with "gibberish" in another profile would be a solution so that it is no problem to send that profile to you if the customer cares about their privacy.

Furthermore the support response times via e-mail were frustrating after it had gone back and forth for a while:
- 8.1. and 15.1.2019 (another as I didn't get any response) -> 15.1.2019
- 16.1. and 29.1.2019 (another as I didn't get any response) -> 29.1.2019
- 22.4.2019 -> 25.4.2019
- I gave up as after 7 e-mails (the first 2 had fast response times) it seemingly was going nowhere. Since then I only used the Android version and thought I wasted money for the PC version which had worked for the first years.
- On Wednesday I sent another e-mail as I thought it's not right that I paid for this software (even bought updates in the hopes it would fix it) and have to figure it out by myself. I got an automatic reply that there will be a response within 1 or 2 business days. I haven't had any response.

Probably you were becoming just as frustrated with the problem as me. I totally understand that.
Furthermore maybe the support person was on holiday or was sick. But shouldn't then the ticket go to somebody else?
However other customers then might give up on you and will avoid your products in the future. Some will want their money back.

As a last suggestion. Maybe setting up an issue tracker like (i.e. via GitHub) would be helpful. Then it easy to keep track between bugs and future requests and if they are solved/rejected and so on.
Inside Google Groups this just goes under. At least in my opinion.

Best regards,
Dan

[Stéph]

Dan, It sounds to me like you would be a perfect person to have on the beta testing team! I wish I understood how to do the analysis you described there.

Just one point - The MLO developers do run some issue tracking, on Jira. I'm not convinced Jira has the best system for prioritisation of issues - New issues or ideas don't get seen or promoted because the items with the most votes sit at the top of the list gathering more votes.

Others have noticed that issue reports and feature requests often don't get a response, until they are fixed. We just have to trust that the MLO team will take the best ideas and most urgent issues and focus on them.

[Stéph]

Correction (pointed out to me by Dwight): Jira doesn't have the voting thing - I was confusing that with User Voice, which MLO was using for gathering users' suggestions and which has a flawed voting system for checking popularity of the posted suggestions. 

Jira is used by the MLO team for bug prioritisation and tracking, but it's probably not the main system they're using because updates on it are pretty infrequent nowadays.

[Andrey Tkachuk (MLO)]

Hi Dan,

Andrey is here, the developer. First of all, let me say that I am very sorry to hear that you had issues with wifi sync.

You are right that such issues are very hard to deal with, especially if you cannot reproduce the problem. From our experience in most cases, it is related to the network or firewall configuration issues.

>This was noticeable as the same names were existing multiple
>times but had added "1", "11" and "2" at the end.
>I myself have never created or modified any contexts or flags.

This usually happens if, for example, you create a new data file which already contains default contexts/flags. After that, you sync this file to your file on the Cloud or through wifi and since you already have such contexts they are renamed during sync to avoid duplication. Our tests show that this case is handled normally by MLO. For the app they are different contexts and it should not be a problem for the sync.

>The solution was simply to remove all of the duplicated entries.
>Afterwards the sync was fine again.

This is very interesting. We did not hear about such causes of the sync issues. In our tests even if contexts are renamed to avoid duplication the sync works as expected. I would like to investigate this case in more details.

>There is no way of downloading older versions via the homepage, or is there?

Some old versions are available on the site, but you can always ask me to send you an older version.

>My goal is simply to give suggestions to make it easier to solve such issues in the future :-)

Thank you. This is very much appreciated.

>In cases where I could not figure out a customer problem
>I created a special debug version with extensive logging

Yes sure we have a different level of logging for debugging and release versions and we send debug version to users in some cases. We also have a closed team of beta testers who always run the beta version with extended logs so that we can identify the issues before the public release.

>So maybe a mechanism to override the title and notes with "gibberish" in another
>profile would be a solution so that it is no problem to send that profile to you if
>the customer cares about their privacy.

Yes sure, thanks for suggesting. We do have such a thing in MLO from the very beginning. So I would appreciate it if you contacted us on sup...@mylifeorganized.net again and help us to reproduce the problem with your data file which which is obfuscated by you before sending. We will provide all the details. I hope you still have an old backup file where you could reproduce the sync issue.

>Probably you were becoming just as frustrated with the problem as me.
>I totally understand that.

Thanks for that Dan. Yes, the support agents just did not know what to answer. They were waiting for analysis from the developers (and from me personally) and as you know this is usually a bottleneck. I am sorry about that. With your help, however, I think we could now analyze the issue again more effectively to avoid it in future for your and for others.

>Maybe setting up an issue tracker like (i.e. via GitHub) would be helpful.

Yes, we do have issue tracking internally which is used for the development and communication between support team and developers. I will see how we could improve this part as well, thank you.

Thank you Dan one more time for the very detailed message and for the suggestions. Also, I would like to invite you to the team of the beta testers. This team always has the latest version with new features. I personally have closer communications with beta testers so that we select new features to develop and high priority fixes. I would love to see you on this team. Let me know if you are interested  :-)

--
Sincerely,
Andrey Tkachuk, CEO and Founder
www.mylifeorganized.net

[Dan]

Hi Stéph and Andrey, thank you to you both for your answers!

Regarding the issue tracking. I thought that maybe a publicly available issue tracker would be good.
This way it would be transparent to the customers what the status of certain issues are.
Furthermore they could for instance vote so that you can see what the main problems/wishes of them are.
However maybe this would need to be restricted so that it does not become chaotic.
But roughly seeing through this Google group I would have thought that it mainly should be well mannered :-)
I use Jira at work and this would be all possible with it. However to my knowledge it's very (!) expensive with lots of users.
That is why I suggested GitHub ;-)

Thank you for the offer regarding beta testing. What exactly would that mean for me? Are there some kind of expectations from me if I join (i.e. tasks / response times)?
If yes, then don't get me wrong but I would kindly decline. With work and private tasks I wouldn't like to be bound to have to do something.
Please also don't get the next question wrong. How stable are these beta versions? I wouldn't like to be scratching my hairs out like I did with the WiFi sync ;-)
Could I have beta and official both installed (just in case)?

In regards to the bug. I could be totally wrong but my guess is that it were the flags. Meaning the old and new design. The rest is just text information which shouldn't be any problem.
I will try to find a backup of the previous state. I believe I should have one. Sorry, if it takes me a while. This weekend I am very busy and during the weekdays I hardly find time after work.
I will send it to you. Should I send it as a new separate e-mail or answer my last one?

Have a nice weekend!

Best regards,
Dan

[Dwight]

Dan, I recommend participating in the beta group. There's no minimum participation requirement. Beta releases tend towards very good stability, the downside is that sometimes the UI for new features can be clumsy when they first appear. But I find it great to have the opportunity to influence the design as these issues get resolved. Jira is used here in a way that compromises some of the benefit and some of the ease-of-use in order to constrain the cost - Will share details if you are interested and if you reply to me off-list at mloduard [at] dwightarthur.us

[Dan]

I have sent a reply to my last support email with the 2 backup files (Desktop and Android). All tasks are deleted.

Thank you Dwight for the information.

Then yes please. I would be interested in joining the beta group.

Regarding Jira. You have your internal workflow which works for you. I don't want to question that :-)

Even though a publicly available issue tracker could be beneficial to the customers having a further bug tracker would make it a confusing muddle for you.

[Andrey Tkachuk (MLO)]

Hi Dan,

Thanks for your answer.

>What exactly would that mean for me? Are there some kind of expectations
>from me if I join (i.e. tasks / response times)?

Sure there are no tasks or response time :) You just install the builds from the beta (not public) upgrade line and work with MLO as you usually do. When I release new features I just ask the team to try them and report in the group findings/suggestions if any. So the beta version usually has more features and you have an opportunity to influence how these functions appear to the public.

There are only selected people on the team who showed they are interested in MLO improvements. This is why I invited you :)

>How stable are these beta versions?

I do not remember any critical issues with betas. In 99.9% it is as stable as public. You may ask the beta team. But anyway as you know backup is always a very good thing.

>Could I have beta and official both installed (just in case)?

Windows - yes, Android - no.

I will send the invitation shortly.

Thanks,
Andrey.

[Stéph]

I can confirm that I've been a beta tester since 2012 (how time flies!) and, so far, have never had a problem attributable to running a beta copy. I've had one corrupted file a long time ago (which I was able to fix by looking through the xml file in notepad), but that won't have been due to any beta testing activity at the time.