Bug? Exported items count lower than the restored items count

130 views
Skip to first unread message

nsms

unread,
May 29, 2024, 3:23:12 PMMay 29
to Session Buddy Discussion
Hi! I converted the v3 database into that json file needed for restoring the items into v4, by using your tool:

https://sessionbuddy.com/v4-migration/

, then I took these steps that led me to noticing that the exported items count is lower than restored items count:

1. I had 1260 collections, and 1225 history events, as you can see  in the image below, totaling 2485 items.

ApplicationFrameHost_OuUlD28eEF.png

2. I went to Settings and changed History to keep at most 2000 events.
chrome_r7GM7RAPnH.png

3. Upon restoring the json file into v4, I got 2484 items, so 1 less item, as it should have been 2485 items.

ApplicationFrameHost_SEIIKV026P.png
All of them got successfully restored.
ApplicationFrameHost_VBPebRfjkB.png

4. Now, when I went to use the Export function, I got only 2449 items.

chrome_Vhcndd63V9.png

Upon closer analysis of the resulting json file due to exporting, I noticed all 1260 colletions were kept, as it was normal, but instead of having 1225 history events, I ended up having only 1189 history events, so 36 history events were missing. Hence 2449 exported items vs 2485 restored items.

I also tested with the 500 and 1000 history size, getting 1760 items (1260 collections + 500 history events), respectively 2260 items (1260 collections + 1000 history events). So everything worked correctly for these 2 options. Only the 2000 history size seems to be having issues, 1260 collection + 1189 history events. So it's stuck at 1189 for some reason, instead of using 2000. For my case it should've gone for the maximum value, 1225.

Session Buddy Support

unread,
May 29, 2024, 4:02:44 PMMay 29
to sessionbud...@googlegroups.com
Could it be that some of those history events had dates that were beyond the age limit? What was "Remove events older than" set to?

Hans


--
You received this message because you are subscribed to the Google Groups "Session Buddy Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sessionbuddy-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sessionbuddy-discuss/73e70a12-a081-43a3-84cd-b74f16fdf276n%40googlegroups.com.

nsms

unread,
May 30, 2024, 3:54:37 AMMay 30
to Session Buddy Discussion

"Remove events older than" was set to "10 years". I also made some modifications to the extension, that would let me use these settings:
chrome_lMUpTggPhw.png
chrome_ZKRiHzG9Fz.png
notepad++_vYpP6Fs94c.png
chrome_B8NcD21ePH.png
notepad++_ZBH4qqzd8w.png

With the "Remove events older than" option set to "1.000.000 years" and with "Keep at most" set to "1 million events" I would still get 2449 items instead of 2485 items .

I'm going to investigate deeper, and try to analyze anything related to dates. I've noticed that some IDs use unix timestamps. I noticed when I converted some of the IDs to human-readable date and time the year would be 2034, and I never had the PC date set to that year, if I recall correctly.

Could it be for those history entries to have been removed because the sessions contained links related only to Session Buddy?

What about these:
chrome_UEddDZBIRc.png
chrome_DqQSXeFr6V.png
I'm not sure about what kind of storage it's being referred to, but currently, I have about 100 GB free on the C drive, where Google Chrome runs from.

nsms

unread,
May 30, 2024, 7:00:22 AMMay 30
to Session Buddy Discussion
I managed to retrieve the "Generated" and "Created" values from the backup json file from v3 for both Previous and Saved Sessions:

https://pastebin.com/raw/1fgBzwrS

There seem to be no anomalies for Previous Sessions , oldest year being 2021 and newest being 2024.
Saved Sessions have a couple of anomalies regarding year 1970, where the range is 1970 - 2024. Even with that, restoring works fine.

Those IDs that I mentioned about earlier it seems the they are not unix timestamps, for v3 json backup files, as in:

         "windows": [
            {
               "alwaysOnTop": false,
               "focused": true,
               "height": 1022,
               "id": 227457093,
               "incognito": false,
               "left": -8,
               "state": "maximized",
               "tabs": [


So no year 2034.

nsms

unread,
Jun 1, 2024, 2:16:14 PMJun 1
to Session Buddy Discussion
I found what was triggering this behavior. I don't know about export, but upon restore/import, duplicates are removed from collections and history.

I made this python script to detect duplicates for both collections and history:

https://pastebin.com/UFK6e3JZ

For my case, the code would return this:

Processing 'collections' section:
Total IDs in JSON using list (including duplicates): 1262
Total unique IDs in JSON using set (excluding duplicates): 1260
Total duplicate IDs: 1
Duplicate IDs and their counts (in order of appearance):
1. ID: AAAAAAAAAAAAAAAAAAAAAAAAAAAA, Count: 3


Processing 'history' section:
Total IDs in JSON using list (including duplicates): 1224
Total unique IDs in JSON using set (excluding duplicates): 1189
Total duplicate IDs: 34
Duplicate IDs and their counts (in order of appearance):
1. ID: 0000000000001, Count: 2
2. ID: 0000000000002, Count: 2
3. ID: 0000000000003, Count: 2
4. ID: 0000000000004, Count: 2
5. ID: 0000000000005, Count: 2
6. ID: 0000000000006, Count: 2
7. ID: 0000000000007, Count: 2
8. ID: 0000000000008, Count: 2
9. ID: 0000000000009, Count: 2
10. ID: 0000000000010, Count: 2
11. ID: 0000000000011, Count: 2
12. ID: 0000000000012, Count: 2
13. ID: 0000000000013, Count: 2
14. ID: 0000000000014, Count: 2
15. ID: 0000000000015, Count: 2
16. ID: 0000000000016, Count: 2
17. ID: 0000000000017, Count: 2
18. ID: 0000000000018, Count: 2
19. ID: 0000000000019, Count: 2
20. ID: 0000000000020, Count: 2
21. ID: 0000000000021, Count: 2
22. ID: 0000000000022, Count: 2
23. ID: 0000000000023, Count: 2
24. ID: 0000000000024, Count: 2
25. ID: 0000000000025, Count: 2
26. ID: 0000000000026, Count: 2
27. ID: 0000000000027, Count: 2
28. ID: 0000000000028, Count: 2
29. ID: 0000000000029, Count: 2
30. ID: 0000000000030, Count: 2
31. ID: 0000000000031, Count: 2
32. ID: 0000000000032, Count: 2
33. ID: 0000000000033, Count: 2
34. ID: 0000000000034, Count: 3

For collections I manually edited the json file to make one session show up for 3 times, to observe how collections duplicates would behave with restore/import.

For privacy reasons I edited the IDs.

Could you please add an option to keep the duplicates, as at times they are really useful as a reference for finding sessions in proximity that would have to deal with information related to the information in those duplicate sessions?

Mohammad Noor

unread,
Jun 3, 2024, 11:10:31 AMJun 3
to Session Buddy Discussion
I totally agree with this, this would explain why I lost around 20 sessions as well. Would be nice to have an option to keep duplicates for sure. 

nsms

unread,
Jun 4, 2024, 1:12:15 PMJun 4
to Session Buddy Discussion

Hopefully tomorrow I can do some additional research, and if it's gonna be successful, I'm gonna make a tool for keeping duplicates, as I have some ideas.

Ismail

unread,
Jun 15, 2024, 9:00:21 AMJun 15
to Session Buddy Discussion
Any update on this Hans? I am having the same issue and would love if we had an option to keep our duplicates when converting the database file.

Mohammad Noor

unread,
Jun 21, 2024, 1:37:02 PM (12 days ago) Jun 21
to Session Buddy Discussion
I second this! Would love to get an update on this problem here. 

Session Buddy Support

unread,
Jun 25, 2024, 2:29:39 PM (8 days ago) Jun 25
to sessionbud...@googlegroups.com
> duplicates are removed from collections and history

This should never be a problem for Import since it generates new ids for collections and ignores history.

For Restore, duplicate ids will result in colliding collections/events being overwritten.

I would never expect to see a v4 export containing duplicate ids unless it was manually edited. A bug causing this is highly unlikely.

For a v3 export or migration tool output, it's a slightly different story:

While I would likewise not expect a duplicate collection id to appear, it is slightly less impossible than for v4 exports. If so, it would have been a bug in v3. Have you observed duplicate collection ids in this type of file? If so, can you provide details.

As you have observed, duplicate event ids are known to occur in v3 exports or migration tool output due to a v3 bug, although rare and expected to correspond to an identical set of windows/tabs. Have you observed dupe event ids with different sets of windows/tabs?

To conclude:

Assumption 1the only observed duplicates have been events sourced from "Previous Sessions" of a v3 database (v3 export file or migration tool output) and duplicates have identical window/tab content
Assumption 2: given assumption 1, collection/event duplicates do not pose a problem

If either of these assumptions is incorrect, please let me know.

Not sure if this helps, but here's a breakdown of the logic:

IMPORT from v4 export and migration tool restore file

Collections

Generate a new unique id for each collection (and every subsequent attempt to import the same collection).

No collisions with existing or other importing collections are expected.

Events

(ignored)

RESTORE from v4 export and migration tool restore file

Collections

id is copied as-is.

Collections (existing or within the restore file) with colliding ids are overwritten.

Events

id (a UNIX timestamp) is copied as-is.

Events (existing or within the restore file) with colliding ids are overwritten.


IMPORT from v3 export

Collections

Generate a new unique id for each collection (and every subsequent attempt to import the same "Saved Session").

No collisions with existing or other importing collections are expected.

Events

(ignored)

RESTORE from v3 export

Collections

id is taken from the "Saved Session" gid field.

If there is no gid or if gid is > 40 chars in length, generate a new unique id and keep this id consistent across multiple attempts to restore the same "Saved Session".

Collections (existing or within the restore file) with colliding ids are overwritten.

Events

id (a UNIX timestamp) is taken from the "Previous Session" generated field.

If generated is empty, use created.

If created is empty, use the current time.

Events (existing or within the restore file) with colliding ids are overwritten.




Reply all
Reply to author
Forward
0 new messages