Connecting perkeep to video server
196 views
Skip to first unread message
dx25
Dec 19, 2018, 6:51:30 PM12/19/18
to Perkeep
Hi,
I have been using the android app on Perkeep to upload photos and videos from my phone to a perkeepd instance I run locally. I'm also running Plex Media Server on that same machine, and would like it to serve up the videos I've added to Perkeep. My first thought was to run pk-mount as a service, pointing Plex there, and hopefully everything would just work. However, I can't figure out how to access content uploaded from the phone via pk-mount.
Ideally I would create a search that finds all videos (e.g. "format:mp4", although this appears not to be supported now), and then be able to mount that search directly in pk-mount. Is something like this possible?
Thanks
Dave
Mathieu Lonjaret
Dec 19, 2018, 7:26:57 PM12/19/18
to per...@googlegroups.com
On Thu, 20 Dec 2018 at 00:51, dx25 <dfrib...@gmail.com> wrote:
>
> Hi,
Hello,
> I have been using the android app on Perkeep to upload photos and videos from my phone to a perkeepd instance I run locally. I'm also running Plex Media Server on that same machine, and would like it to serve up the videos I've added to Perkeep. My first thought was to run pk-mount as a service, pointing Plex there, and hopefully everything would just work.
Yes, in theory that should work.
> However, I can't figure out how to access content uploaded from the phone via pk-mount.
Have you seen https://perkeep.org/cmd/pk-mount/ ?
What step are you blocked at exactly?
> Ideally I would create a search that finds all videos (e.g. "format:mp4", although this appears not to be supported now), and then be able to mount that search directly in pk-mount. Is something like this possible?
No, I don't think you can do that kind of thing for now. You can only
control whether you want to only mount a subset via the second
argument of pk-mount. So what you can actually do pretty easily, is to
gather (after you searched for them) all your videos under a common
root. Then you mount that very root with pk-mount.
hth,
Mathieu
> You received this message because you are subscribed to the Google Groups "Perkeep" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to perkeep+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
> Hi,
Hello,
> I have been using the android app on Perkeep to upload photos and videos from my phone to a perkeepd instance I run locally. I'm also running Plex Media Server on that same machine, and would like it to serve up the videos I've added to Perkeep. My first thought was to run pk-mount as a service, pointing Plex there, and hopefully everything would just work.
> However, I can't figure out how to access content uploaded from the phone via pk-mount.
What step are you blocked at exactly?
> Ideally I would create a search that finds all videos (e.g. "format:mp4", although this appears not to be supported now), and then be able to mount that search directly in pk-mount. Is something like this possible?
control whether you want to only mount a subset via the second
argument of pk-mount. So what you can actually do pretty easily, is to
gather (after you searched for them) all your videos under a common
root. Then you mount that very root with pk-mount.
hth,
Mathieu
> You received this message because you are subscribed to the Google Groups "Perkeep" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to perkeep+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
dx25
Dec 20, 2018, 9:55:01 AM12/20/18
to Perkeep
Sorry, that was worded poorly. My hope was to avoid creating a folder, and then every time phone uploads something, manually adding that to the folder. Sounds like for now that's the way to do it though.
Mathieu Lonjaret
Dec 20, 2018, 10:11:21 AM12/20/18
to per...@googlegroups.com
It does not necessarily have to be manual though. The simplest (but
least efficient) way I can imagine to automatize it would be to make a
program monitoring new things being added to your Perkeep (a camtool
search call for recent items, being run regularly, for example), and
when detecting when one of these new items is a video, it would add it
to the mounted camliRoot.
You could also add a custom handler which does that within Perkeep. In
the same way the cond blobserver (pkg/blobserver/cond) is used to add
a blob to the index only if it is a schema blob. But that is a bit
more involved than the above.
> --
least efficient) way I can imagine to automatize it would be to make a
program monitoring new things being added to your Perkeep (a camtool
search call for recent items, being run regularly, for example), and
when detecting when one of these new items is a video, it would add it
to the mounted camliRoot.
You could also add a custom handler which does that within Perkeep. In
the same way the cond blobserver (pkg/blobserver/cond) is used to add
a blob to the index only if it is a schema blob. But that is a bit
more involved than the above.
dx25
May 8, 2019, 4:56:24 PM5/8/19
to Perkeep
I finally got around to looking at this video server idea again -- as Mathieu suggested, I created a folder in web UI and moved all video files to the folder. Now when pk-mounted, the new folder is not visible in "roots" unless I manually set the camliRoot attribute to it, and none of the videos are visible within unless I set, say, a camliPath:filename.mp4 attribute on the folder permanode. Mathieu, you mentioned adding a custom handler. As I look through the source, I see a number of other blobservers, but I have no clue how to use them or what they are for. I'm assuming that was how were suggesting I deal with this camliRoot/camliPath problem?
The other idea I had was an approach similar to how <pkmount>/recent folder is working. Within that folder, I can see filenames for some recently added stuff, apparently without the need for these redundant camliPath: attributes (since the fileName is stored with the blob content for the videos). Was curious to see how this actually populates the /recent folder, but was unable to locate the implementation. Any idea where I should look for it? As I was saying in my original post, it seems like what I'd really want here is to write my own alternative implementation of /recent (called, e.g. /videos), that makes all uploaded videos available. I was able to successfully add a "is:video" search parameter that uses the mime type prefix, but that's as far as I've gotten.
So, any ideas on what direction to go next?
On Thursday, December 20, 2018 at 9:11:21 AM UTC-6, mathieu.lonjaret wrote:
It does not necessarily have to be manual though. The simplest (but
least efficient) way I can imagine to automatize it would be to make a
program monitoring new things being added to your Perkeep (a camtool
search call for recent items, being run regularly, for example), and
when detecting when one of these new items is a video, it would add it
to the mounted camliRoot.
You could also add a custom handler which does that within Perkeep. In
the same way the cond blobserver (pkg/blobserver/cond) is used to add
a blob to the index only if it is a schema blob. But that is a bit
more involved than the above.
On Thu, 20 Dec 2018 at 15:55, dx25 <df...@gmail.com> wrote:
>
> Sorry, that was worded poorly. My hope was to avoid creating a folder, and then every time phone uploads something, manually adding that to the folder. Sounds like for now that's the way to do it though.
>
>
>
> On Wednesday, December 19, 2018 at 6:26:57 PM UTC-6, mathieu.lonjaret wrote:
>>
>> On Thu, 20 Dec 2018 at 00:51, dx25 <df...@gmail.com> wrote:
>> >
>> > Hi,
>>
>> Hello,
>>
>> > I have been using the android app on Perkeep to upload photos and videos from my phone to a perkeepd instance I run locally. I'm also running Plex Media Server on that same machine, and would like it to serve up the videos I've added to Perkeep. My first thought was to run pk-mount as a service, pointing Plex there, and hopefully everything would just work.
>>
>> Yes, in theory that should work.
>>
>> > However, I can't figure out how to access content uploaded from the phone via pk-mount.
>>
>> Have you seen https://perkeep.org/cmd/pk-mount/ ?
>> What step are you blocked at exactly?
>>
>> > Ideally I would create a search that finds all videos (e.g. "format:mp4", although this appears not to be supported now), and then be able to mount that search directly in pk-mount. Is something like this possible?
>>
>> No, I don't think you can do that kind of thing for now. You can only
>> control whether you want to only mount a subset via the second
>> argument of pk-mount. So what you can actually do pretty easily, is to
>> gather (after you searched for them) all your videos under a common
>> root. Then you mount that very root with pk-mount.
>>
>> hth,
>> Mathieu
>>
>> > You received this message because you are subscribed to the Google Groups "Perkeep" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an email to perkeep+u...@googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "Perkeep" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to perkeep+unsubscribe@googlegroups.com.
Eric Drechsel
May 8, 2019, 6:26:31 PM5/8/19
to per...@googlegroups.com
Hi Dave,
On Wed, May 8, 2019 at 1:56 PM dx25 <dfrib...@gmail.com> wrote:
I finally got around to looking at this video server idea again -- as Mathieu suggested, I created a folder in web UI and moved all video files to the folder. Now when pk-mounted, the new folder is not visible in "roots" unless I manually set the camliRoot attribute to it, and none of the videos are visible within unless I set, say, a camliPath:filename.mp4 attribute on the folder permanode. Mathieu, you mentioned adding a custom handler. As I look through the source, I see a number of other blobservers, but I have no clue how to use them or what they are for. I'm assuming that was how were suggesting I deal with this camliRoot/camliPath problem?
The other idea I had was an approach similar to how <pkmount>/recent folder is working. Within that folder, I can see filenames for some recently added stuff, apparently without the need for these redundant camliPath: attributes (since the fileName is stored with the blob content for the videos). Was curious to see how this actually populates the /recent folder, but was unable to locate the implementation.
The mapping of nodes to filenames happens starting at https://github.com/perkeep/perkeep/blob/7f17c0483f2e86575ed87aac35fb75154b16b7f4/pkg/fs/recent.go#L104
Any idea where I should look for it? As I was saying in my original post, it seems like what I'd really want here is to write my own alternative implementation of /recent (called, e.g. /videos), that makes all uploaded videos available. I was able to successfully add a "is:video" search parameter that uses the mime type prefix, but that's as far as I've gotten.So, any ideas on what direction to go next?
fs.RecentDir is a good template for a search-backed directory. It seems like it might be appropriate to make RecentDir just an instance of a fs.SearchDir type. RecentDir doesn't have any tests but SearchDir should.
I'd want to see a new "/search" node that implements Lookup() to take a filename-escaped search string and return a new SearchDir node similar to how fs.root implements hash-based permanode lookup here:https://github.com/perkeep/perkeep/blob/7f17c0483f2e86575ed87aac35fb75154b16b7f4/pkg/fs/root.go#L136) since that would provide users with a way to perform searches through the filesystem.
To unsubscribe from this group and stop receiving emails from it, send an email to perkeep+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/1b38a8f6-d118-4951-9ae7-dbd87ffe1a07%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Mathieu Lonjaret
May 9, 2019, 5:32:36 PM5/9/19
to per...@googlegroups.com
Hello,
On Thu, 9 May 2019 at 00:26, Eric Drechsel <er...@pdxhub.org> wrote:
>
> Hi Dave,
>
> On Wed, May 8, 2019 at 1:56 PM dx25 <dfrib...@gmail.com> wrote:
>>
>> I finally got around to looking at this video server idea again -- as Mathieu suggested, I created a folder in web UI and moved all video files to the folder. Now when pk-mounted, the new folder is not visible in "roots" unless I manually set the camliRoot attribute to it, and none of the videos are visible within unless I set, say, a camliPath:filename.mp4 attribute on the folder permanode. Mathieu, you mentioned adding a custom handler. As I look through the source, I see a number of other blobservers, but I have no clue how to use them or what they are for. I'm assuming that was how were suggesting I deal with this camliRoot/camliPath problem?
>>
>> The other idea I had was an approach similar to how <pkmount>/recent folder is working. Within that folder, I can see filenames for some recently added stuff, apparently without the need for these redundant camliPath: attributes (since the fileName is stored with the blob content for the videos). Was curious to see how this actually populates the /recent folder, but was unable to locate the implementation.
As Eric said, if you want to see how the magic of pk mount works, it's
all in pkg/fs. But honestly if you want things to show up as files in
pk mount, I maintain that it's probably easier to do a bit of glueing
and scripting on top of things, to add the relevant camliRoots and
camliPaths to your permanodes, than to hack a new "magic directory" in
pkg/fs. Dealing with fuse stuff isn't easy imho.
> Mathieu, you mentioned adding a custom handler. As I look through the source, I see a number of other blobservers, but I have no clue how to use them or what they are for. I'm assuming that was how were suggesting I deal with this camliRoot/camliPath problem?
The other idea I had mentioned earlier is also harder than the very
first option, but probably not as hard as inventing new things in
pkg/fs. But it still requires you to understand what typically happens
to a blob when Perkeep receives it, with regard to how the conditional
blobserver only passes it to the indexer if it is a schema blob, then
how the indexer treats it depending on what kind of mutation it is,
etc. My very rough idea was that you could add your own custom piece
in the middle that does the job of creating the camliPaths and
camliRoots somewhere along the line, when e.g. the indexer sees it is
a video file. But again, that is pretty involved too, and wouldn't be
a very adaptable/reusable mechanism, so all in all, probably not a
very good idea.
> To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/CAGTxtWiNUbSG_m4mLYqHsPfinxaVqUGB29G2J%2Bjr5VZuYrUTYA%40mail.gmail.com.
On Thu, 9 May 2019 at 00:26, Eric Drechsel <er...@pdxhub.org> wrote:
>
> Hi Dave,
>
> On Wed, May 8, 2019 at 1:56 PM dx25 <dfrib...@gmail.com> wrote:
>>
>> I finally got around to looking at this video server idea again -- as Mathieu suggested, I created a folder in web UI and moved all video files to the folder. Now when pk-mounted, the new folder is not visible in "roots" unless I manually set the camliRoot attribute to it, and none of the videos are visible within unless I set, say, a camliPath:filename.mp4 attribute on the folder permanode. Mathieu, you mentioned adding a custom handler. As I look through the source, I see a number of other blobservers, but I have no clue how to use them or what they are for. I'm assuming that was how were suggesting I deal with this camliRoot/camliPath problem?
>>
>> The other idea I had was an approach similar to how <pkmount>/recent folder is working. Within that folder, I can see filenames for some recently added stuff, apparently without the need for these redundant camliPath: attributes (since the fileName is stored with the blob content for the videos). Was curious to see how this actually populates the /recent folder, but was unable to locate the implementation.
all in pkg/fs. But honestly if you want things to show up as files in
pk mount, I maintain that it's probably easier to do a bit of glueing
and scripting on top of things, to add the relevant camliRoots and
camliPaths to your permanodes, than to hack a new "magic directory" in
pkg/fs. Dealing with fuse stuff isn't easy imho.
> Mathieu, you mentioned adding a custom handler. As I look through the source, I see a number of other blobservers, but I have no clue how to use them or what they are for. I'm assuming that was how were suggesting I deal with this camliRoot/camliPath problem?
first option, but probably not as hard as inventing new things in
pkg/fs. But it still requires you to understand what typically happens
to a blob when Perkeep receives it, with regard to how the conditional
blobserver only passes it to the indexer if it is a schema blob, then
how the indexer treats it depending on what kind of mutation it is,
etc. My very rough idea was that you could add your own custom piece
in the middle that does the job of creating the camliPaths and
camliRoots somewhere along the line, when e.g. the indexer sees it is
a video file. But again, that is pretty involved too, and wouldn't be
a very adaptable/reusable mechanism, so all in all, probably not a
very good idea.
dx25
May 21, 2019, 5:34:41 PM5/21/19
to Perkeep
I've been trying out Eric's magic directory idea, using the "recent" and "at" implementations as a guide. I was able to get it working (sort of) -- within the new "search" subfolder, I can do something like cd "after:\"10-01-2017\" and is:image", and get a directory back with the right results, except that it seems to max out at 201 results for some reason.
>>> > To unsubscribe from this group and stop receiving emails from it, send an email to per...@googlegroups.com.
>>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Perkeep" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to per...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/1b38a8f6-d118-4951-9ae7-dbd87ffe1a07%40googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> best, Eric
> eric.pdxhub.org
>
> --
> You received this message because you are subscribed to the Google Groups "Perkeep" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to per...@googlegroups.com.
Mathieu Lonjaret
May 21, 2019, 5:55:34 PM5/21/19
to per...@googlegroups.com
Probably because search queries have a default max number of returned
results. You should be able to get an unlimited amount by specifying a
Limit of -1 in your query. See
https://perkeep.org/pkg/search#SearchQuery.
> To unsubscribe from this group and stop receiving emails from it, send an email to perkeep+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/3f330c68-61b2-48a3-9c80-0489a839d503%40googlegroups.com.
results. You should be able to get an unlimited amount by specifying a
Limit of -1 in your query. See
https://perkeep.org/pkg/search#SearchQuery.
> To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/3f330c68-61b2-48a3-9c80-0489a839d503%40googlegroups.com.
dx25
May 21, 2019, 6:20:20 PM5/21/19
to Perkeep
Nice, now it seems to be working.
Eric Drechsel
May 23, 2019, 4:08:48 AM5/23/19
to per...@googlegroups.com
Super cool!
I'd be interested to get something like that merged. Would you be up
for submitting a patch (https://perkeep.org/code#contributing)? Even
if you don't want to finish it up, it's a starting point.
I'm pretty familiar with the fs module and would be happy to review.
I noticed there's support for persisted searches (namedSearch in
https://github.com/perkeep/perkeep/blob/7f17c0483f2e86575ed87aac35fb75154b16b7f4/pkg/search/predicate.go#L695),
would be cool to make them show up as search/name/ dirs, although
there's no UI currently for creating them.
> To unsubscribe from this group and stop receiving emails from it, send an email to perkeep+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/3ffb2f72-8c98-4e8e-90f4-69e514009184%40googlegroups.com.
I'd be interested to get something like that merged. Would you be up
for submitting a patch (https://perkeep.org/code#contributing)? Even
if you don't want to finish it up, it's a starting point.
I'm pretty familiar with the fs module and would be happy to review.
I noticed there's support for persisted searches (namedSearch in
https://github.com/perkeep/perkeep/blob/7f17c0483f2e86575ed87aac35fb75154b16b7f4/pkg/search/predicate.go#L695),
would be cool to make them show up as search/name/ dirs, although
there's no UI currently for creating them.
> To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/3ffb2f72-8c98-4e8e-90f4-69e514009184%40googlegroups.com.
dx25
May 23, 2019, 10:44:06 AM5/23/19
to Perkeep
Ok.
For the video server, I was able to get minidlna to see my is:video folder, and dlna clients seem to play the videos just fine. However, I did have to run minidlna under the same user as perkeep mount. Seems there are fuse options that can be set in Bazil.org/fuse to allow the mount to be accessible to other users. Also no thumbnails right now, and only filename based -- would be nice if there was a way to pass along DLNA metadata but not sure what's involved there. I might make a write-up about the whole setup, it took quite a bit of fiddling to get up and running.
Eric Drechsel
May 23, 2019, 8:23:56 PM5/23/19
to per...@googlegroups.com
On Thu, May 23, 2019 at 7:44 AM dx25 <dfrib...@gmail.com> wrote:
>
> Ok.
>
> For the video server, I was able to get minidlna to see my is:video folder, and dlna clients seem to play the videos just fine. However, I did have to run minidlna under the same user as perkeep mount. Seems there are fuse options that can be set in Bazil.org/fuse to allow the mount to be accessible to other users. Also no thumbnails right now, and only filename based -- would be nice if there was a way to pass along DLNA metadata but not sure what's involved there.
I don't see a way for Perkeep to pass metadata to DLNA clients without
>
> Ok.
>
> For the video server, I was able to get minidlna to see my is:video folder, and dlna clients seem to play the videos just fine. However, I did have to run minidlna under the same user as perkeep mount. Seems there are fuse options that can be set in Bazil.org/fuse to allow the mount to be accessible to other users. Also no thumbnails right now, and only filename based -- would be nice if there was a way to pass along DLNA metadata but not sure what's involved there.
being a DLNA server itself, short of directory heirarchy through
minidlna.
> To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/0f6ead92-6fcd-4f25-ae51-4bf23b1a0387%40googlegroups.com.
dx25
May 28, 2019, 12:09:05 PM5/28/19
to Perkeep
Here's the review. Needs improvement, any suggestions are appreciated.
Reply all
Reply to author
Forward
0 new messages