Local file storage
51 views
Skip to first unread message
Justin Nesselrotte
May 21, 2014, 4:43:15 AM5/21/14
to botq...@googlegroups.com
Hello everyone! I'm happy to inform you that one of the long awaited features of local file storage (server side, instead of S3) has been implemented! I'm really excited about this, as it allows hackerspaces to host an internal BotQueue instance without the cost of S3.
If you'd like to figure out how to do that, and have a BotQueue instance, read on! If you don't have a BotQueue instance, don't worry! You can use the main BotQueue.com (which is currently out of date), or you can email me off list if you want to set up an instance for your organization!
(Note, it is suggested that no bots attempt to communicate with the server at this point, otherwise you'll end up with files in s3 that aren't in your local storage. This guide also only applies to people running 0.5X-dev and not currently to the master branch. That will change when 0.5 is released)
First, add these two lines to your config.php file:
define("STORAGE_METHOD", "LocalFile");
define("STORAGE_PATH", "/var/www/storage");
The storage path has to be readable and writable by whatever user apache is running under. In my case it was www-data, so I just changed the user and group of that folder to www-data and the permissions to user read and write only.
If you've been running BotQueue before, look into the scripts directory for the convert-s3-to-local.php file. If you run it on the command line, with the lines added to your config, it will start to download all of your s3 files to the directory you specified. Now when you let bots connect again, it will be accessing the local files instead of the S3 files.
Since I was working with this anyway, I cleaned up a lot of the S3 code anyway, and update it, so that will hopefully reduce an amount of issues that were previously had.
Note that this is still a relatively new addition, and I'm trying to iron out all of the bugs. I'm technically not entirely done with everything I'm going to do with file storage, but this works now. If there are any issues, please let me know.
If you'd like to figure out how to do that, and have a BotQueue instance, read on! If you don't have a BotQueue instance, don't worry! You can use the main BotQueue.com (which is currently out of date), or you can email me off list if you want to set up an instance for your organization!
(Note, it is suggested that no bots attempt to communicate with the server at this point, otherwise you'll end up with files in s3 that aren't in your local storage. This guide also only applies to people running 0.5X-dev and not currently to the master branch. That will change when 0.5 is released)
First, add these two lines to your config.php file:
define("STORAGE_METHOD", "LocalFile");
define("STORAGE_PATH", "/var/www/storage");
The storage path has to be readable and writable by whatever user apache is running under. In my case it was www-data, so I just changed the user and group of that folder to www-data and the permissions to user read and write only.
If you've been running BotQueue before, look into the scripts directory for the convert-s3-to-local.php file. If you run it on the command line, with the lines added to your config, it will start to download all of your s3 files to the directory you specified. Now when you let bots connect again, it will be accessing the local files instead of the S3 files.
Since I was working with this anyway, I cleaned up a lot of the S3 code anyway, and update it, so that will hopefully reduce an amount of issues that were previously had.
Note that this is still a relatively new addition, and I'm trying to iron out all of the bugs. I'm technically not entirely done with everything I'm going to do with file storage, but this works now. If there are any issues, please let me know.
Philip Booysen
May 21, 2014, 5:40:44 AM5/21/14
to botq...@googlegroups.com
Hi Justin,
You rock and this is awesome!
I will peruse and implement / play around with soon.
Regards,
Philip
--
You received this message because you are subscribed to the Google Groups "BotQueue" group.
To unsubscribe from this group and stop receiving emails from it, send an email to botqueue+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages