Programmable interactions with VisualSFM V0.5.24

[Changchang Wu]

Hi guys,

If you ever want to interact with VisualSFM in a programmable way, try out the socket command interface in VisualSFM V0.5.24. Basically, you will be able to simulate most of the menu clicks by sending the corresponding command through socket communication.

First, you launch VisualSFM in the server mode, for example, "VisualSFM listen 9999" will let VisualSFM launch the GUI and listen to port 9999.

You can then connect form a client, and send commands to VisualSFM. The list of available command <id> can be found in http://ccwu.me/vsfm/ui.ini as menu_<id>.  For example:
    1) "33045 absolute_path/list.txt\n",  open absolute_path/list.txt and load the image list
    2) "33033\n",  do all the feature matching
    3) "33041\n",  reconstruct the sparse model
    4) "33044 absolute_path/result.nvm\n", save the reconstruction to absolute_path/result.nvm

    5) "33045 absolute_path/one.nvm\n", open one.nvm

    6) "33471 absolute_path/two.nvm\n", run dense reconstruction, and save absolute_path/two.nvm

Edit: it is best to use absolute path. For slash, it is best to use \ in Windows but / for other OS.

See the documentation at http://ccwu.me/vsfm/changelist.html and run "VisualSFM help" for more details. Enjoy

Changchang Wu

[Curtis Boirum]

Hi Changchang Wu,

Thank you so much for this feature, it is excellent for automation and I am attempting to make extensive use of it for near real time localization of a moon rover at CMU (using 1 640x480 image every 5 seconds). I am having issues with some of the menu functions not taking arguments such as "#menu_33299 Set Fixed Calibration" and "#menu_33498 Compute Sequence Match". Command calls are accepted by VSFM but the argument is not recognized (the calibration is not set and sequence matching is only done with a value of 2 when I specefied 4). I am able to run image load, sift, and resume reconstruction with no problem, and of course image load takes the file path argument just fine.

I have attached my script (which works great if you don't need to specify fixed cal or sequence matching params).

Best,
Curtis Boirum

[Changchang Wu]

Use comma instead of space for calibration input : 265.6548,324.6482,265.7165,232.9243,0

[Jiang Dong]

Hi Changchang Wu,

 Firstly, thank you for posting the excellent software!

 I am working on a localization system which is based on VisualSFM. Every time there is a new localization request with a picture, I will store the picture path as 'source.nvm.txt' and then run the command line 'VisualSFM sfm+resume source.nvm target.nvm' . After that I can get the location of the new picture in target.nvm.

The problem I met is that every time I get a request I need to re-run the command line to load the matching records, which is time consuming when the model is large. Following is part of the log when I was testing a model with 2 030 pictures (size 2048x1152). It took 30 seconds to load the matching records. 

I wonder if there is any way to pre-load these records and run the sift extraction, matching parts directly when new requests coming either in command line mode or in socket command mode?

**************************************

Loading matching records ...<141, 10K> <200, 20K> <245, 30K> <283, 40K> <317, 50K> <348, 60K> <376, 70K> <402, 81K> <427, 91K> <450, 101K> <472, 111K> <493, 121K> <513, 131K> <533, 142K> <552, 152K> <570, 162K> <588, 173K> <605, 183K> <622, 193K> <638, 203K> <654, 214K> <670, 224K> <685, 234K> <700, 245K> <715, 255K> <729, 266K> <743, 276K> <757, 286K> <771, 297K> <784, 307K> <797, 318K> <810, 328K> <823, 339K> <836, 349K> <848, 359K> <860, 370K> <872, 380K> <884, 391K> <896, 401K> <908, 412K> <919, 422K> <930, 432K> <941, 443K> <952, 453K> <963, 464K> <974, 474K> <985, 485K> <996, 496K> <1006, 506K> <1016, 516K> <1026, 526K> <1036, 537K> <1046, 547K> <1056, 558K> <1066, 568K> <1076, 579K> <1086, 590K> <1096, 601K> <1106, 612K> <1116, 623K> <1125, 633K> <1134, 643K> <1143, 653K> <1152, 664K> <1161, 674K> <1170, 685K> <1179, 695K> <1188, 706K> <1197, 717K> <1206, 727K> <1215, 738K> <1224, 749K> <1233, 760K> <1242, 771K> <1251, 783K> <1259, 793K> <1267, 803K> <1275, 813K> <1283, 823K> <1291, 833K> <1299, 844K> <1307, 854K> <1315, 865K> <1323, 875K> <1331, 886K> <1339, 897K> <1347, 907K> <1355, 918K> <1363, 929K> <1371, 940K> <1379, 951K> <1387, 962K> <1395, 973K> <1403, 984K> <1411, 996K> <1419, 1007K> <1427, 1018K> <1434, 1028K> <1441, 1038K> <1448, 1049K> <1455, 1059K> <1462, 1069K> <1469, 1079K> <1476, 1090K> <1483, 1100K> <1490, 1110K> <1497, 1121K> <1504, 1131K> <1511, 1142K> <1518, 1152K> <1525, 1163K> <1532, 1174K> <1539, 1185K> <1546, 1195K> <1553, 1206K> <1560, 1217K> <1567, 1228K> <1574, 1239K> <1581, 1250K> <1588, 1261K> <1595, 1272K> <1602, 1284K> <1609, 1295K> <1616, 1306K> <1623, 1317K> <1630, 1329K> <1637, 1340K> <1644, 1352K> <1651, 1363K> <1658, 1375K> <1665, 1386K> <1671, 1396K> <1677, 1407K> <1683, 1417K> <1689, 1427K> <1695, 1437K> <1701, 1447K> <1707, 1457K> <1713, 1468K> <1719, 1478K> <1725, 1488K> <1731, 1499K> <1737, 1509K> <1743, 1519K> <1749, 1530K> <1755, 1540K> <1761, 1551K> <1767, 1562K> <1773, 1572K> <1779, 1583K> <1785, 1594K> <1791, 1604K> <1797, 1615K> <1803, 1626K> <1809, 1637K> <1815, 1648K> <1821, 1658K> <1827, 1669K> <1833, 1680K> <1839, 1691K> <1845, 1702K> <1851, 1714K> <1857, 1725K> <1863, 1736K> <1869, 1747K> <1875, 1758K> <1881, 1770K> <1887, 1781K> <1893, 1792K> <1899, 1804K> <1905, 1815K> <1911, 1826K> <1917, 1838K> <1923, 1849K> <1929, 1861K> <1935, 1873K> <1941, 1884K> <1947, 1896K> <1953, 1908K> <1959, 1919K> <1965, 1931K> <1971, 1943K> <1977, 1955K> <1983, 1967K> <1989, 1979K> <1995, 1990K> <2001, 2002K> <2006, 2012K> <2011, 2023K> <2016, 2033K> <2021, 2043K> <2026, 2053K> done in 30s

759MB used to store 2059341 matching records;

200 gps coordinates are found

***************************************************

Best regards!

Jiang Dong

[Changchang Wu]

You have to use the socket command interface. Basically you should be able to simulate most GUI interactions, for example, add an image, resume the reconstruction, save file, remove image..

> --
> You received this message because you are subscribed to the Google Groups "VisualSFM" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vsfm+uns...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Heiner Kunkel]

Hi Changchang Wu,

I am working with the client socket.
It is working fine, but I have some problems with loading pictures.
The command #menu_33028    Open+ Image &  SIFT is working: 33028 C:\Users\user\Desktop\Hiwi Job Max\bundler-v0.4-source\examples\ET\et000.jpg
The problem is that I cannot use Less Visualization with this command.
How can I use the #menu_33166    Open+ Multi Images command with more than one picture?
Something like this: 33166 C:\Users\user\Desktop\Hiwi Job Max\osm-bundler\examples\ET/et001.jpg/et002.jpg

Thank you!

Heiner Kunkel

[Changchang Wu]

The socket interface only supports a single argument. You can create a txt  file to contain all the images you want to open,  and use "Load/Add N-View Match" instead.

[Heiner Kunkel]

Thank you, thats working fine!

Another question:
In the version 22 there was a pmvs image reduce setting.
Where can I find this setting in the newer versions?
It was a real useful tool.

Thank you!

[Changchang Wu]

The size reducing function got dropped in the latest version. It seems easier to resize the images before the sfm?

[Dan Monaghan]

Hi Changchang,

Just trying develop my multi machine vsfm server to use your sockets function, I’m trying with SocketTest, plus the code here http://www.linuxhowtos.org/C_C++/socket.htm and I’m not having much luck.  I start on my server machine VisualSFM listen 2017 , then send a command such as ui_dark_background 1 and on the server I see Received: ui_dark_background 1, but nothing changes, I can’t use it to quit either, so I’m wondering what I may be doing wrong?  Also porting my multi machine vsfm server to cpp, my first big c project...

Cheers,

Dan

[Changchang Wu]

You need to send the command with an ID that corresponds to the menu..For the dark background switch you need to send "33451"...

Look for #menu_(id) for each menu item.

[Nick Rhinehart]

Hi Changchang,

I'm trying to add images incrementally, e.g.

33166 /usr0/home/nrhineha/Desktop/couch/000.jpg
33166 /usr0/home/nrhineha/Desktop/couch/001.jpg

33166 /usr0/home/nrhineha/Desktop/couch/002.jpg

but only the first command completes successfully.

I've tried putting these filenames in separate [image_basename].txt files:

list_0.txt contents:

absolute_path/000.jpg

list_1.txt contents:

absolute_path/001.jpg

etc.

Again, only the first image is loaded successfully.

Any idea what might be the problem?

I'm using the python interface I posted earlier today to send these commands, although that shouldn't be the issue.

Here's the VSFM log (which doesn't reflect the fact that it didn't load images [001] - [004] correctly):

"""

Received: 33166 /usr0/home/nrhineha/Desktop/couch/000.jpg

0: 000

image_size: 3264x2448

#sift count: 2804

Loading image pixel data ...done in 0s

Received: 33166 /usr0/home/nrhineha/Desktop/couch/001.jpg

Loading image pixel data ...done in 0s

Received: 33166 /usr0/home/nrhineha/Desktop/couch/002.jpg

Loading image pixel data ...done in 0s

Received: 33166 /usr0/home/nrhineha/Desktop/couch/003.jpg

Loading image pixel data ...done in 0s

Received: 33166 /usr0/home/nrhineha/Desktop/couch/004.jpg

Loading image pixel data ...done in 0s

[Changchang Wu]

I just found that there is a bug with 33166.

You can use  33202 with the .txt file as an workaround for now.

[Nick Rhinehart]

Hi Changchang,

Thanks for the bugfix!

Another question, a bit tangential:

Is it possible with the current interface to load an NVM, and then input a query image and run a partial bundle adjustment on the edges between the new image and other images, leaving the underlying model unmodified? e.g. localizing a new view within the current model. I understand this is probably a feature request ... :)

[Changchang Wu]

Sounds like It can be done in two steps:

1. Match with the new image, either using "Compute Missing Matches" or  "Compute Specified Matches"

2. Shift + "Reconstruct Resume" will keep existing cameras unmodified (points may still change).

[Nick Rhinehart]

Awesome! Thanks for the reply. It worked. I can now automate new camera localization within a known bundle via the socket interface, but I suppose I'll have to write out the entire NVM to retrieve the new camera params & positions..? 

--

Nick Rhinehart

PhD student

Robotics Institute

Carnegie Mellon University

[Allan Jabri]

Hey Nick,

Is this the case? Do you have to write out the NVM?

Thanks

Allan

[Changchang Wu]

You can use "Save Current Camera", which will save a compact NVM without points