New Advanced Harbour Google Groups Messages Search engine on harbour.wiki

564 views
Skip to first unread message

Eric Lendvai

unread,
Mar 25, 2019, 2:46:44 AM3/25/19
to Harbour Users

Web scrapped more than 54,000 messages from both users and developers Harbour Google Groups.


You can now search a decade of messages, in just a few seconds in both groups, at the same time.


This is a free tools that can help you find solutions faster.


I will shortly create article on harbour.wiki, detailing all the steps needed to web scrape the data yourself.

It takes more than 20 hours and creates 150 Mb tables (DBF/FPT)!


To make it easy for everyone, I added an advanced search feature on the wiki.


https://harbour.wiki/index.asp?page=PublicSearchGoogleGroups


A button is also available on the home page of harbour.wiki


GoogleMessage.png


Diego Fazio

unread,
Mar 25, 2019, 7:02:55 AM3/25/19
to Harbour Users
Great JoB!

Serge Girard

unread,
Mar 25, 2019, 10:35:58 AM3/25/19
to Harbour Users
Works great !

Thanks,

Serge

Op maandag 25 maart 2019 07:46:44 UTC+1 schreef Eric Lendvai:

Massimo Belgrano

unread,
Mar 25, 2019, 11:08:43 AM3/25/19
to harbou...@googlegroups.com
Good  and Thanks,

Mail priva di virus. www.avast.com

--
--
You received this message because you are subscribed to the Google
Groups "Harbour Users" group.
Unsubscribe: harbour-user...@googlegroups.com
Web: http://groups.google.com/group/harbour-users

---
You received this message because you are subscribed to the Google Groups "Harbour Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to harbour-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
Massimo Belgrano
Delta Informatica S.r.l. (Cliccami per scoprire 

hua

unread,
Mar 26, 2019, 3:40:37 AM3/26/19
to Harbour Users
This is useful. Thanks Eric!
Message has been deleted
Message has been deleted

Eric Lendvai

unread,
Mar 27, 2019, 3:32:10 AM3/27/19
to Harbour Users
Sorry for the spelling mistake, "scrapped" should be "scraped", single letter "P.

Clippero

unread,
Mar 27, 2019, 12:58:27 PM3/27/19
to Harbour Users
Genial !



El lunes, 25 de marzo de 2019, 3:46:44 (UTC-3), Eric Lendvai escribió:

Eric Lendvai

unread,
Mar 27, 2019, 1:02:17 PM3/27/19
to Harbour Users
Thank you for everyone's positive feedback!

Mel Smith

unread,
Mar 27, 2019, 4:38:49 PM3/27/19
to Harbour Users
Hi Eric:

   Please feel free to 'scrape' my web site (www.whosaway.com) and tell me and the rest of the world what you find.

   *You* have my explicit permission to do so.

   btw, I wish that I could understand the complex steps that you posted. But, it is a bit too much for my brain to process :((
 
- Mel Smith

Eric Lendvai

unread,
Mar 28, 2019, 10:14:38 AM3/28/19
to Harbour Users
Hello Mel,

Thank you for your offer/request. 
I am not certain what kind of information to scrape for.

In the meantime I can add your web site to the links area of harbour.wiki.
Could you write me a title and explanation for the link to your site?

Thanks, Eric 

Eric Lendvai

unread,
Mar 28, 2019, 10:26:52 AM3/28/19
to Harbour Users
Just pushed an update to make searching for Author's behave the same way as "Keywords", meaning you can enter multiple text and the will all be searched for, used in a ".AND." way.
So searching "lend eri" will find all Message from me.

Also changed the default to search in the Message Text and Topic Title at the same time.

Eric Lendvai

unread,
Mar 29, 2019, 1:36:01 PM3/29/19
to harbou...@googlegroups.com
Thanks Chris.
I will try to find a way to let you select your date format. Probably would require you to web push subscriber to remember your setting.
Eric

On Mar 28, 2019 11:33 PM, "Chris_D" <Chr...@gmx.co.uk> wrote:

Excellent.  Thank you.


Can we get rid of the ghastly U.S. date format in the timestamps?

--
--
You received this message because you are subscribed to the Google
Groups "Harbour Users" group.

Web: http://groups.google.com/group/harbour-users

---
You received this message because you are subscribed to the Google Groups "Harbour Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to harbour-users+unsubscribe@googlegroups.com.

Eric Lendvai

unread,
Mar 30, 2019, 6:11:29 AM3/30/19
to Harbour Users
Just updated harbour.wiki to detect new Google Group messages every 10 minutes. 
Meaning any messages older than 10 minutes in the User or Developer Google Groups should included in any advanced queries.


On Sunday, March 24, 2019 at 11:46:44 PM UTC-7, Eric Lendvai wrote:

Eric Lendvai

unread,
Apr 2, 2019, 3:28:39 AM4/2/19
to Harbour Users
Added feature to allow to select Date and Time format other than US. Still does not "remember" selected setting if you leave the search page. 


On Sunday, March 24, 2019 at 11:46:44 PM UTC-7, Eric Lendvai wrote:

Daniel Aisenberg

unread,
Nov 16, 2022, 4:16:27 PM11/16/22
to harbou...@googlegroups.com
brilliant. many thanks

--
--
You received this message because you are subscribed to the Google
Groups "Harbour Users" group.

Web: http://groups.google.com/group/harbour-users

---
You received this message because you are subscribed to the Google Groups "Harbour Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to harbour-user...@googlegroups.com.

Daniel Aisenberg

unread,
Nov 16, 2022, 4:20:02 PM11/16/22
to harbou...@googlegroups.com
may i ask you which tool do you use to perform such a web scraping in gmail ?

Eric Lendvai

unread,
Nov 24, 2022, 2:35:02 AM11/24/22
to Harbour Users
Hello Daniel,

But at some point google forced to be logged in during scrapping and I did not make a work around it yet.
Hope this helps. 
If you find a solution, please share it back and I will update the repo and article.
Eric

Daniel Aisenberg

unread,
Nov 24, 2022, 4:37:52 PM11/24/22
to harbou...@googlegroups.com
Many thanks


---
You received this message because you are subscribed to the Google Groups "Harbour Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to harbour-user...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages