How to spit Firefox bookmark urls into a single text file

[Banter]

Is there a way to spit Firefox bookmarks to a text list of URLs on Linux?

I have about 50 to 100 bookmarks under a Firefox bookmark folder.
That works fine - but I want to spit out the list to a text file.

Is there a way to do that?

[crankypuss]

Easiest way I know of is to start by exporting the bookmarks to an html
file.

[Andy K]

I looked for a Linux way, but it was rather complicated using sed etc.

This is for windows in case you use it to.

It takes any .html file and extracts only the urls.

http://www.focalmedia.net/urlextract.html

[Banter]

On Thu, 14 Mar 2013 16:31:22 +0100, FredW wrote:

> [ Not annouced follow-up to an unknown group removed ]
> http://kb.mozillazine.org/Places.sqlite

Darn newserver forces a single followup newsgroup.

$ sudo updatedb; locate places.sqlite
=> /home/usr1/.gnome2/gnome-web-photo/places.sqlite
=> /home/usr1/.gnome2/gnome-web-photo/places.sqlite-journal
=> /home/usr1/.mozilla/firefox/amen8qeb.default/places.sqlite
=> /home/usr1/.mozilla/firefox/amen8qeb.default/places.sqlite-shm
=> /home/usr1/.mozilla/firefox/amen8qeb.default/places.sqlite-wal
=> /home/usr1/.thunderbird/7mxej0uw.default/places.sqlite

$ file /home/usr1/.mozilla/firefox/amen8qeb.default/places.sqlite
=> places.sqlite: SQLite 3.x database, user version 12

$ cat /home/usr1/.mozilla/firefox/amen8qeb.default/places.sqlite
Yikes! Not a text file!

Reading again, I see it can be edited using a freeware addon:
https://addons.mozilla.org/en-US/firefox/addon/sqlite-manager

Once installed:
Firefox: Tools->SQLite Manager

There is no "File->Open"; there is only:
SQLite Manager: Database->New Database
But, the desired sqlite file is locked (probably because Firefox has
it locked - but SQLite Manager runs in Firefox so I'll make a copy).

$ copy ~/.mozilla/firefox/amen8qeb.default/places.sqlite /tmp

SQLite Manager: Database->New Database->/tmp/places.sqlite
Select Folder to Place the DB file: /tmp/db -> Open

Wow. This is not a pretty interface at all!
http://www1.picturepush.com/photo/a/12413404/img/12413404.png

Maybe we need to export to a text file?
SQLite Manager: Table->Export Table->CSV->/tmp/db/sqlite_master.csv
http://www4.picturepush.com/photo/a/12413427/img/12413427.png

$ cat /tmp/db/sqlite_master.csv
=> "type","name","tbl_name","rootpage","sql"

Drat. This will take some practice to get to work.

Q: Any idea what step I'm missing?

[Banter]

Getting closer. At least I have a text file now ...

$ mkdir /tmp/db
$ copy ~/.mozilla/firefox/amen8qeb.default/places.sqlite /tmp/db/.
Firefox: Tools->SQLite Manager
SQLite Manager: Database->Connect Database->/tmp/db/places.sqlite
SQLite Manager: Table->Export Table->CSV->Name of the Table->moz_bookmarks
http://www2.picturepush.com/photo/a/12413510/img/12413510.png
$ cat /tmp/db/moz_bookmarks.csv

Yuk. Getting better though, because at least this is a text (comma separated
values) file. It's just got a LOT of garbage in it ... so I need to look
deeper to figure out how to output JUST the URLs that are bookmarked.

[Allodoxaphobia]

And then 'view' the html file with `lynx' (?):

$ lynx -dump -nolist exportedfile.html

Or, maybe you _do_ wan the URL list. I.e., drop the " -nolist ".

Just shotgunning it now. Not at a linux workstation to test it...

Jonesy
--
Marvin L Jones | jonz | W3DHJ | linux
38.24N 104.55W | @ config.com | Jonesy | FreeBSD & OS/2
* Killfiling google & XXXXbanter.com: jonz.net/ng.htm

[Whiskers]

I don't know of a specific tool for the job, but if the browser can
'export' the bookmarks to an HTML file, then you could use "awk" to extract
the URLs to a text file. That's as far as my knowledge of awk goes!

--
-- ^^^^^^^^^^
-- Whiskers
-- ~~~~~~~~~~

[Cousin Stanley]

Banter wrote:

> Is there a way to spit Firefox bookmarks
> to a text list of URLs on Linux ?

> ....

You might try using the sqlite3
command line interface ....

The following works for me
under debian linux ....

Fnd the ff bookmarks database ....

$ locate places.sqlite

I found it at ....

~/.mozilla/firefox/*.default/places.sqlite

Then ....

$ sqlite3 ~/.mozilla/firefox/*.default/places.sqlite
>
> .output /home/your_user_name/some/dir/ff_bookmarks.txt
>
> .width 128
>
> select url from moz_places order by url ;
>
> ctrl-d to quit


The > are sqlite prompt characters


--
Stanley C. Kitching
Human Being
Phoenix, Arizona

[Kirk_Von_Rockstein]

This works for me...thanks

[Jasen Betts]

> http://[crapware.kom]

By what measure is that simpler?


--
⚂⚃ 100% natural

--- news://freenews.netfront.net/ - complaints: ne...@netfront.net ---

[Bit Twister]

On 16 Mar 2013 02:25:45 GMT, Jasen Betts wrote:

> By what measure is that simpler?

Depends on your desires and mode of operation.

Makes it easy to bring bookmarks from work and home and merge them
together.

Since I have a few bookmarks

$ wc -l < /local/doc/urls
3485

I happen to append "! some key words" to end of url so I can find
something"

when I need something like documentation for bash I run a script to find them

$ urls doc bash
http://www.howtoforge.com/detailed-error-handling-in-bash ! document sqlite trap
http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html ! basic shell bash doc
http://gentoo-wiki.com/MAN_bash ! documentation
http://cfaj.freeshell.org/shell ! bash script tips usage doc
http://tldp.org/LDP/abs/html/index.html ! bash script advanced documentation
http://mywiki.wooledge.org/BashFAQ/050 ! bash script variable expansion doc

[Banter]

On Thu, 14 Mar 2013 20:47:53 +0100, FredW wrote:

> You could export the bookmarks as a (bookmarks).html file.
> In about:config (search for 'bookmark')
> - browser.bookmarks.autoExportHTML

1. I opened Firefox & typed 'about:config' into the URL bar.
2. I searched for 'browser.bookmarks.autoExportHTML'
3. Doubleclicking changed it from 'False' to 'True'.
... hmm ... now what? ...
4. Googling for 'browser.bookmarks.autoExportHTML' reveals:
http://kb.mozillazine.org/Browser.bookmarks.autoExportHTML
Which says the bookmarks.html file is located here:
/home/uname/.mozilla/firefox/GIBBERISH.default/bookmarks.html
5. Since that's a whacky location, more googling finds this:
http://kb.mozillazine.org/Browser.bookmarks.file
Which says we can CHANGE the default location with:
New->String->browser.bookmarks.file = /tmp/bookmark.htm

That seemed to work! Thanks.
Firefox automatically creates the /tmp/bookmark.htm file now.

Unfortunately, the syntax of that bookmark.htm file is
absolutely horrid; so I will explore the sed solutions
provided in order to gracefully cull out JUST the URLs!

[Banter]

On Thu, 14 Mar 2013 20:04:29 +0000, J G Miller wrote:
> Bookmarks -> Show All Bookmarks
> Export Bookmarks
> sed -ne 's|.*\<A HREF="\(http[^"]*\)".*|\1|p' bm.htm > url.txt

Wow. That worked wonders! Thanks.

Putting both ideas together, here's a sequence tutorial:

Title: How to save Firefox bookmarks to URL lists on Linux:
0. Manually export bookmarks using this sequence:
Firefox: Bookmarks->Show All Bookmarks
Show All Bookmarks: Import & Backup->Export Bookmarks to HTML
File = /tmp/bm.htm
1. Or, automatically export bookmarks using this sequence:
Firefox: about:config->browser.bookmarks.autoExportHTML=True
Creates: ~/.mozilla/firefox/XXX.default/bookmarks.html
2. Optionally, change the default location to something sane:
Firefox: about:config->browser.bookmarks.file=/tmp/bm.htm
3. Convert that bookmark HTML file to a list of URLs:
sed -ne 's|.*\<A HREF="\(http[^"]*\)".*|\1|p' bm.htm > bm.txt

I'm having trouble creating the alias (due to the quotes):
$ vi ~/.bash_aliases
alias bm='sed -ne \'s|.*\<A HREF="\(http[^"]*\)".*|\1|p\' /tmp/bm.htm > /tmp/bm.txt'

Note: I tried backslashing the initial & ending tick, but that also failed.
How do you properly escape the ticks in order to make a bash alias?

[Cousin Stanley]

> This works for me...thanks

You're welcome.

Since the ff bookmarks are stored in an sqlite database,
the most direct and simplest route to get them out
and into a .txt file seems to me using an sql select statement
instead of first exporting to .html from a browser
followed by a subsequent process to turn them back
into plain text ....

[Banter]

On Sat, 16 Mar 2013 20:03:30 +0000, J G Miller wrote:
> alias bm="sed -ne 's|.*\<A HREF=\"\(http[^\"]*\)\".*|\1|p' /tmp/bm.htm > /tmp/bm.txt"

Worked like a charm! Thanks!

I really don't use shell scripts (I'm not a coder), but this will work
for what I need, which is simply a way to strip all the URLs out of
the bookmark files!

Thanks everyone. Mission accomplished!

[Banter]

On Thu, 14 Mar 2013 19:59:32 +0000, Allodoxaphobia wrote:

> $ lynx -dump -nolist exportedfile.html
> Or, maybe you _do_ wan the URL list. I.e., drop the " -nolist ".

Wow, that worked like a charm!

I had never heard of "lynx", but sure enough, it's there:
$ man lynx
Lynx is a fully-featured World Wide Web (WWW) client for users
running cursor-addressable, character-cell display devices ...

Lynx works just fine to list JUST the bookmarked URLs:
0. Firefox: Bookmarks->Show All Bookmarks
1. Show All Bookmarks: Import & Backup->Export Bookmarks to HTML
2. File = /tmp/bookmark.htm
3. $ lynx -dump /tmp/bookmark.htm

[Banter]

On Sat, 16 Mar 2013 03:06:12 +0000, Bit Twister wrote:

> I happen to append "! some key words" to end of url

> so I can find something" ...

I like the idea of keeping a separate URL file.
Does the bang (!) syntax work in HTML though?

[Banter]

On Fri, 15 Mar 2013 03:44:51 +0000, Cousin Stanley wrote:

> $ sqlite3 ~/.mozilla/firefox/*.default/places.sqlite
> .output /home/your_user_name/some/dir/ff_bookmarks.txt
> .width 128
> select url from moz_places order by url ;
> ctrl-d to quit

$ which sqlite3
/usr/bin/sqlite3

$ man sqlite3
sqlite3 is a terminal-based front-end to the SQLite library
that can evaluate queries interactively and display the results ...

$ sudo updatedb; locate places.sqlite

/home/username/.mozilla/firefox/atrn8qsb.default/places.sqlite

$ cp ~/.mozilla/firefox/*.default/places.sqlite .

$ sqlite3 places.sqlite
SQLite version 3.6.20
Enter ".help" for instructions
Enter SQL statements terminated with a ";"

sqlite> .output result.txt
sqlite> .width 128
sqlite> select url from moz_places order by url ;
Error: file is encrypted or is not a database

[Banter]

On Sun, 17 Mar 2013 00:14:36 +0000, Cousin Stanley wrote:

> Since the ff bookmarks are stored in an sqlite database,
> the most direct and simplest route to get them out
> and into a .txt file seems to me using an sql select statement
> instead of first exporting to .html from a browser
> followed by a subsequent process to turn them back
> into plain text ....

I agree.

But, unfortunately, the commands report an error which I'm not
sure why I get and you don't get:

$ cp ~/.mozilla/firefox/*.default/places.sqlite .

$ sqlite3 places.sqlite

==> SQLite version 3.6.20
==> Enter ".help" for instructions
==> Enter SQL statements terminated with a ";"

sqlite> .output result.txt
sqlite> .width 128

sqlite> select url from moz_places order by url ;
==> Error: file is encrypted or is not a database

[Bit Twister]

No idea. Create a test account, say junk, log into junk and see what
you can see. :-D

[Cousin Stanley]

Banter wrote:

>
> But, unfortunately, the commands report an error
> which I'm not sure why I get and you don't get:

Maybe path errors if your places.sqlite data base
is installed in a different location from the one
that I use ????

> $ cp ~/.mozilla/firefox/*.default/places.sqlite .
>
> $ sqlite3 places.sqlite
> ==> SQLite version 3.6.20
> ==> Enter ".help" for instructions
> ==> Enter SQL statements terminated with a ";"
>
> sqlite> .output result.txt
> sqlite> .width 128
> sqlite> select url from moz_places order by url ;
> ==> Error: file is encrypted or is not a database
>

Since you're only reading from the data base
and not writing to it, making a copy would be
an extra non-essential task but should still
work ok ....

You might try the file command on both
the original path returned from locate places.sqlite
and on the copy you made which should verify
if the files are sqlite3 data bases ....

$ file places.sqlite

places.sqlite: SQLite 3.x database, user version 12

If the file command verifies that places.sqlite
is an sqlite3 data base then check for a list
of tables in the data base and for the schema infomation
about the moz_places table ....

$ sqlite3 places.sqlite
> .tables
............
> .schema moz_places

[Banter]

On Mon, 18 Mar 2013 17:51:49 +0000, Cousin Stanley wrote:
> $ file places.sqlite
> places.sqlite: SQLite 3.x database, user version 12

> $ sqlite3 places.sqlite
> > .tables
> ............
> > .schema moz_places

$ file places.sqlite
==> places.sqlite: SQLite 3.x database, user version 12

$ sqlite3 places.sqlite
==> SQLite version 3.6.20
==> Enter ".help" for instructions
==> Enter SQL statements terminated with a ";"

sqlite> .tables

==> Error: file is encrypted or is not a database

sqlite> .schema moz_places

[Cousin Stanley]

Banter wrote:

> $ sqlite3 places.sqlite
> ==> SQLite version 3.6.20
> ==> Enter ".help" for instructions
> ==> Enter SQL statements terminated with a ";"
> sqlite> .tables
> ==> Error: file is encrypted or is not a database
> sqlite> .schema moz_places
> ==> Error: file is encrypted or is not a database

The only obvious difference I'm seeing
is the difference in our sqlite3 versions
but it would suprise me if that would be problematic ....

yours : 3.6.20

mine : 3,7.13

# --------------------------------------------------

$ cd ~/.mozilla/firefox/*.default

$ ls -1 *.sqlite
addons.sqlite
chromeappsstore.sqlite
content-prefs.sqlite
cookies.sqlite
downloads.sqlite
extensions.sqlite
formhistory.sqlite
permissions.sqlite
places.sqlite
search.sqlite
signons.sqlite
urlclassifier2.sqlite
urlclassifier3.sqlite
webappsstore.sqlite

$ file places.sqlite
places.sqlite: SQLite 3.x database, user version 12

$ which sqlite3
/usr/local/bin/sqlite3

$ sqlite3 places.sqlite
-- Loading resources from /home/sk/.sqliterc
SQLite version 3.7.13
Enter ".help" for instructions

Enter SQL statements terminated with a ";"
sqlite>

sqlite> .tables
moz_anno_attributes moz_favicons moz_keywords
moz_annos moz_historyvisits moz_places
moz_bookmarks moz_inputhistory
moz_bookmarks_roots moz_items_annos
sqlite>
sqlite> .schema moz_places
............

[Banter]

On Mon, 18 Mar 2013, Cousin Stanley wrote:

> The only obvious difference ... is ... in our sqlite3 versions
> yours : 3.6.20, mine : 3,7.13

Hi Cousin Stanley,

Don't worry about that as I already have an answer using the
$ lynx -dump /tmp/bookmark.htm

But, just to compare with your results, here's what I get:
$ ls -l ~/.mozilla/firefox/*.default/*.sqlite
==> addons.sqlite extensions.sqlite signons.sqlite
==> chromeappsstore.sqlite formhistory.sqlite urlclassifier3.sqlite
==> content-prefs.sqlite permissions.sqlite webappsstore.sqlite
==> cookies.sqlite places.sqlite
==> downloads.sqlite search.sqlite

$ file places.sqlite
==> places.sqlite: SQLite 3.x database, user version 12 <== same as yours

$ which sqlite3
==> /usr/bin/sqlite3 <== (yours is in /usr/local/bin/sqlite3)

$ sqlite3 places.sqlite
==> SQLite version 3.6.20
==> Enter ".help" for instructions
==> Enter SQL statements terminated with a ";"

==> sqlite>

.tables
==> Error: file is encrypted or is not a database

Again, no need to go further. I'm not worried because I already
have a decent solution. Thanks for the additional information though!

[J G Miller]

On Tuesday, March 19th, 2013, at 17:13:10h +0000, Banter observed:

> $ sqlite3 places.sqlite
> ==> SQLite version 3.6.20
> ==> Enter ".help" for instructions
> ==> Enter SQL statements terminated with a ";"
> ==> sqlite>
> .tables
> ==> Error: file is encrypted or is not a database

This is very bizarre.

On this system Mint 14 Nadia with Firefox 19.0.2

> file places.sqlite

places.sqlite: SQLite 3.x database, user version 21

> sqlite3 places.sqlite

SQLite version 3.7.13 2012-06-11 02:05:22
Enter ".help" for instructions

Enter SQL statements terminated with a ";"

sqlite> .tables
moz_anno_attributes moz_favicons moz_items_annos
moz_annos moz_historyvisits moz_keywords
moz_bookmarks moz_hosts moz_places
moz_bookmarks_roots moz_inputhistory
sqlite>

What do you see when you use the command

file places.sqlite

It could be that the version of the sqlite3 binary is
not compatible with the sqlite file created by firefox
and that is why you are getting the strange error message.

[Banter]

On Wed, 20 Mar 2013 15:25:42 +0000, J G Miller wrote:

> What do you see when you use the command
> file places.sqlite
> It could be that the version of the sqlite3 binary is
> not compatible with the sqlite file created by firefox
> and that is why you are getting the strange error message.

I'm going to give up on sqlite as I already have an alternative,
in that both lynx and sed work just fine on the bookmarks file.

However, to answer your question:
$ file ~/.mozilla/firefox/*.default/places.sqlite
==> SQLite 3.x database, user version 12

$ sqlite3 --version
==> 3.6.20

$ uname -a
==> Linux 2.6.32-279.19.1.el6.x86_64 ...

$ firefox --version
==> Mozilla Firefox 10.0.12

There's nothing wrong with the places.sqlite as far as I can tell
simply because everything else works just fine other than sqlite3.

[Richard Kettlewell]

J G Miller <mil...@yoyo.ORG> writes:
> It could be that the version of the sqlite3 binary is
> not compatible with the sqlite file created by firefox
> and that is why you are getting the strange error message.

I believe Firefox embeds its own version of sqlite; if so perhaps a more
recent one than the OP is using.

--
http://www.greenend.org.uk/rjk/

[Banter]

On Wed, 20 Mar 2013 15:47:52 +0000, Richard Kettlewell wrote:

> I believe Firefox embeds its own version of sqlite; if so perhaps a more
> recent one than the OP is using.

That could easily be the case as there are very many sqlite packages!

$ yum info sqlite
==> reported two "installed packages:

Name : sqlite
Arch : i686
Version : 3.6.20
Release : 1.el6
Size : 629 k
Repo : installed
From repo : base

Name : sqlite
Arch : x86_64
Version : 3.6.20
Release : 1.el6
Size : 626 k
Repo : installed
From repo : anaconda-CentOS-201112091719.x86_64

$ yum --noplugins --showduplicates --enablerepo \* --disablerepo c6-media,\*-source,\*debug\* provides "*/sqlite3"
==> reported MANY different sqlite3 installation packages!

Just "some" of the possibilities are listed below!
sqlite-3.6.20-1.el6.i686 : Library that implements an embeddable SQL database
Repo : C6.0-base, Filename : /usr/bin/sqlite3

python-2.6.5-3.el6.i686 : An interpreted, interactive, object-oriented
Repo : C6.0-base, Filename : /usr/lib/python2.6/sqlite3

sqlite-tcl-3.6.20-1.el6.x86_64 : Tcl module for the sqlite3 embeddable SQL
Repo : C6.0-base, Filename : /usr/lib64/tcl8.5/sqlite3

etc. (this list went on and on and on)

[Richard Kettlewell]

Banter <ban...@banter.com> writes:
> Richard Kettlewell wrote:

>> I believe Firefox embeds its own version of sqlite; if so perhaps a more
>> recent one than the OP is using.
>
> That could easily be the case as there are very many sqlite packages!
>
> $ yum info sqlite
> ==> reported two "installed packages:
>
> Name : sqlite
> Arch : i686
> Version : 3.6.20
> Release : 1.el6
> Size : 629 k
> Repo : installed
> From repo : base
>
> Name : sqlite
> Arch : x86_64
> Version : 3.6.20
> Release : 1.el6
> Size : 626 k
> Repo : installed
> From repo : anaconda-CentOS-201112091719.x86_64

Those are the same version, built for different architectures.

> $ yum --noplugins --showduplicates --enablerepo \* --disablerepo c6-media,\*-source,\*debug\* provides "*/sqlite3"
> ==> reported MANY different sqlite3 installation packages!
>
> Just "some" of the possibilities are listed below!
> sqlite-3.6.20-1.el6.i686 : Library that implements an embeddable SQL database
> Repo : C6.0-base, Filename : /usr/bin/sqlite3

That’s the command line interface for the version quoted above.

> python-2.6.5-3.el6.i686 : An interpreted, interactive, object-oriented
> Repo : C6.0-base, Filename : /usr/lib/python2.6/sqlite3
>
> sqlite-tcl-3.6.20-1.el6.x86_64 : Tcl module for the sqlite3 embeddable SQL
> Repo : C6.0-base, Filename : /usr/lib64/tcl8.5/sqlite3

Those are the language for Python and Tcl, again for the same version.

There are indeed multiple versions of sqlite (just as with most other
software), but all you’ve found here is multiple reflections of one
particular version.

--
http://www.greenend.org.uk/rjk/

[Richard Kettlewell]

Richard Kettlewell <r...@greenend.org.uk> writes:

> Banter <ban...@banter.com> writes:

>> python-2.6.5-3.el6.i686 : An interpreted, interactive, object-oriented
>> Repo : C6.0-base, Filename : /usr/lib/python2.6/sqlite3
>>
>> sqlite-tcl-3.6.20-1.el6.x86_64 : Tcl module for the sqlite3 embeddable SQL
>> Repo : C6.0-base, Filename : /usr/lib64/tcl8.5/sqlite3
>
> Those are the language for Python and Tcl, again for the same version.

“language bindings”, of course.

--
http://www.greenend.org.uk/rjk/