How to export linux fs using USB MassStorage device
Dan Danillo
I'm using an embedded linux system and want to make a part of the file
system accessible from a Windows System using USB mass storage.
I know about the usb gadget driver g_file_store. This however only
works with devices or files.
As I want to be able to access the mass storage from Windows the
device/partiton or file must be formated in fat.
Is there a solution out there which lets me map a part of the linux fs
(tempfs, jffs2, fuse) as a fat formated block device or file?
Michael Schnell
There is a free EXT2/3 driver for Windows that does work finw with USB
sticks and so it will work with your device, too, if you export a media
that is formatted as EXT2/3.
If you want to avoid any installation on the Windows PC, you need to do
or find a virtualization that simulates an FAT formatted device.
Regarding the simplicity of FAT, I suppose that is quite doable in a
read-only mode, especially if there is only a main directory, no long
file names and the all files are smaller than 512 Bytes.
-Michael
Michael Schnell
512 Byte is nonsense. Without additional complexity, you can offer files
as large as one cluster and you can define the cluster size when doing
this software. IIRC the max cluster size is 64 K.
-Michael
Michael Schnell
problem (in read-only mode). I suppose assigning more clusters than the
file length requires is not really forbidden.
BTW.: regarding the latest patent-claim by Microsoft, you are not
allowed to deal with long file names on FAT drives in an embedded device.
-Michael
Dan Danillo
> or find a virtualization that simulates an FAT formatted device.
>
Hi Michael,
That's exactly what I want to do. Any idea where to start looking for
such a virtualization - and yes, read only is all I need?
As I have not found one yet I'm looking into the possibility of
implementing it myself.
I'm just not sure what the right approach would be. My idea would be
to implement a block device driver.
The so created block device I would provide to the g_file storage
module.
Any suggestions?
Regards,
Dan
Michael Schnell
>
> That's exactly what I want to do. Any idea where to start looking for
> such a virtualization - and yes, read only is all I need
I'm not aware of such a virtualization, maybe you need to do it
yourself. But FAT is not that complicated, so read-only seems quite
doable.
The PC requests blocks of 512 byte sectors and you need to proved them.
You can simulate a constant FAT and an appropriate root directory giving
file (supposedly constant) names, sizes and (constant) start start
cluster numbers. The PC now will request the appropriate sectors
containing the clusters with the file content, the bytes of which you
will fetch from the real files
> As I have not found one yet I'm looking into the possibility of
> implementing it myself.
> I'm just not sure what the right approach would be. My idea would be
> to implement a block device driver.
That might be possible and would allow for using the g_file software you
mentioned to hook same to the USB interface, but I feel that this is
much more complex than handling the appropriate USB read block yourself
(i.e. modifying the g_file software ). (But I might be wrong.)
-Michael
Dan Danillo
<mschnell_at_bschnell_dot...@aol.com> wrote:
>
> I'm not aware of such a virtualization, maybe you need to do it
> yourself. But FAT is not that complicated, so read-only seems quite
> doable.
I see.
Would have been nice if there where a solution available.
>
> That might be possible and would allow for using the g_file software you
> mentioned to hook same to the USB interface, but I feel that this is
> much more complex than handling the appropriate USB read block yourself
> (i.e. modifying the g_file software ). (But I might be wrong.)
>
> -Michael
Well I looked into it and that might be the final solution, but if I
write an own device driver
I only have to worry of the device driver not working.
When modifying g_file I would need to worry about breaking the UBS
part ...
Anyway, thank you for your support,
Dan
Michael Schnell
>
> Well I looked into it and that might be the final solution, but if I
> write an own device driver
> I only have to worry of the device driver not working.
> When modifying g_file I would need to worry about breaking the UBS
> part ...
You are right, but writing a block driver is much more challenging than
writing user software, especially regarding debugging.
I suppose g_file interfaces the block device it is runtime-configured to
use at just a few places. You would replace "write" block-device
access(es) by an error condition and the "read" the block-device
access(es) by a function call to the function you will create. Same gets
a sector number and count and is supposed to fill a buffer with a
sequence of bytes. This can't harm the USB functionality.
This done you might even be inclined to use your function to do the
block driver which of course is the more "clean" solution.
Of course I would be interested to see the source code once you get it
going :)
-Michael
Dan Danillo
<mschnell_at_bschnell_dot...@aol.com> wrote:
> I suppose g_file interfaces the block device it is runtime-configured to
> use at just a few places. You would replace "write" block-device
> access(es) by an error condition and the "read" the block-device
> access(es) by a function call to the function you will create. Same gets
> a sector number and count and is supposed to fill a buffer with a
> sequence of bytes. This can't harm the USB functionality.
device
> Of course I would be interested to see the source code once you get it
> going :)
Well, lets see if I ever get that far ^^
> -Michael
Well it sounds easy as you write it, but creating a fat in the
background - at least some caching for the most important structures
is required - and give the right block - created on the fly - back
when asked for the block number seems not an easy task to me.
So I plan to first create a few simple things as user apps:
- simple static fat which can be mounted using a loop device
- simple app read a directory structure, convert it into static fat,
mountable using loop
Now write a simple block device driver (module) which builds upppon
simple app 2
challenges here might be:
- access directory using vfs access inside the kernel instead of user
space
- get the blocks dynamicly created
This block device should be mountable as fat
If that is done I only need to load g_file_storage with the device
- sounds easy ... right ^^
Dan
Michael Schnell
> Well it sounds easy as you write it, but creating a fat in the
> background - at least some caching for the most important structures
> is required - and give the right block - created on the fly - back
> when asked for the block number seems not an easy task to me.
I don't see what you mean here. For a read-only thingy you don't "create
ab fat in the background", but you simulate one. You provide the PC with
a completely constant file-system definition and FAT (only one is
necessary) and a directory that only is dynamic regarding the file site.
Now the PC will request blocks with the files' content according to the
FAT information you provided. Your program will read these blocks from
the real files-
>
> So I plan to first create a few simple things as user apps:
> - simple static fat which can be mounted using a loop device
I don't see why you need a loop device - or anything additional to the
modified g_file software at all.
> - simple app read a directory structure, convert it into static fat,
> mountable using loop
???
>
> Now write a simple block device driver (module) which builds upppon
> simple app 2
> challenges here might be:
> - access directory using vfs access inside the kernel instead of user
> space
> - get the blocks dynamicly created
> This block device should be mountable as fat
>
> If that is done I only need to load g_file_storage with the device
> - sounds easy ... right ^^
To me that sounds rather complicated (compared to just modifying the
g_file program to simulate an FAT device for the PC.
-Michael
Michael Schnell
-Michael
Michael Schnell
of a simple (i.e. static, sauf than the file content), I'd estimate some
500 lines of easily testable C code. A draft should be doable in a day.
-Michael
Michael Schnell
If the PC request sector 0 it gets a file system description. IIRC same
defines the sector where the FAT starts, where the root dir starts and
where the file area starts. I'd do an FAT 16 simulation using 32 K
clusters. This information is static (always the same).
If the PC requests a sector of the root file system it gets the
description of the files provided. This information is static, only the
files' sizes can be set according to the current situation. The file's
start sectors can be something like 0x100, 0x200, 0x300, ... I suppose a
single sector (normal is 512 bytes) will be sufficient (or do you plan
for a huge number of files)
If the PC request a sector from the FAT you can calculate these 512
bytes to represent sector sequences like 0x100, 0x101, 0x102, ...,
0x200, 0x201, 0x202, .... 0x300, 0x301, 0x302, .... the sequence length
done according to the max file length you want to support. This
information is static but I suppose it's better to calculate it on
request than to store it somewhere.
If the PC requests a sector (or multiple) from the file area (sector =
file area start sector + cluster * (clustersize / sectorsize) ) you
calculate the cluster no, and thus the file in question and the relative
location of the sector. So you can open the file and read the
appropriate bytes and provide them via USB.
Sounds quite doable to me.
BTW.: Where did you fined the g_file code ? I'd like to take a look at
it but googeling did not show a location.
-Michael
Dan Danillo
<mschnell_at_bschnell_dot...@aol.com> wrote:
> Still ranting on...
Keep doing it ;) - been a bit bussy.
> If the PC requests a sector (or multiple) from the file area (sector =
> file area start sector + cluster * (clustersize / sectorsize) ) you
> calculate the cluster no, and thus the file in question and the relative
> location of the sector. So you can open the file and read the
> appropriate bytes and provide them via USB.
Yep. that's the plan, but you have to create the directory entries for
it.
And they might change - using an arbitrary directory in the Linux file
system as root of the fat.
Of course, using a directory with files only for starters would be a
sensible approach.
> BTW.: Where did you fined the g_file code ? I'd like to take a look at
> it but googeling did not show a location.
>
> -Michael
Its part of the Linux kernel
driver/usb/gadget/file_storage.c
Dan
Michael Schnell
>
> Yep. that's the plan, but you have to create the directory entries for
> it.
> And they might change - using an arbitrary directory in the Linux file
> system as root of the fat.
OK if you don't want to provide just a constant set of files, things get
somewhat more complicated. Nonetheless the files could start at cluster
numbers like 0x100, 0x200, 0x300, ...
But what will be the file names ? Long file names are not allowed
because of the Microsoft patent (TomTom just payed M$ for allowing them
to use this).
>
> Of course, using a directory with files only for starters would be a
> sensible approach.
I feel that a constant set of files would suffice in most application
that provide monitoring an embedded device via the USB File Media interface.
>
> Its part of the Linux kernel
> driver/usb/gadget/file_storage.c
I'll take a look...
I see that it's a driver, not a user land application. This could make
things somewhat more complicated. (1) debugging and (2) I don't know how
to access files from a Kernel driver (should be possible, though).
Ranting along: Maybe you could create a device driver that provides a
block device for g_file to use and provides a simple character device
that a userland daemon can hook to and wait for request from the block
site which it will answer accordingly.
Or you create a modified g_file driver that provides a character device
for that daemon instead of using a block device interface of another
driver.
Maybe this easier than doing everything in a driver.
-Michael
cs_po...@hotmail.com
Create a fat filesystem inside an image file on your linux file
system. Point gadget at this storage space, and it will represent it
to the USB host as a mass storage device containing a fat filesystem.
What you can't do is modify the contents of that file while it's
mounted on another machine as a USB mass storage device - even if it's
mounted read only, the remote machine isn't expecting the contents of
the disk to go and change unexpectedly, as that could break any
caching of data or metadata by the remote system.
I think if you want the embedded device to be able to modify it while
it's mounted, then windows is going to have to see it is a shared file
system, like a network one. To do that, you'd either need to install
drivers on the windows system - for example, gadget's usb network
driver, and then install samba on the embedded device and treat it as
a windows share.
Michael Schnell
> mounted on another machine as a USB mass storage device - even if it's
> mounted read only, the remote machine isn't expecting the contents of
> the disk to go and change unexpectedly, as that could break any
> caching of data or metadata by the remote system.
>
That of course is very true and has not been discussed yet.
If installing something propriety on the Windows system is necessary,
maybe using "serial via USB" might be the easiest way to go.
-Michael
Dan Danillo
> Create a fat filesystem inside an image file on your linux file
> system. Point gadget at this storage space, and it will represent it
> to the USB host as a mass storage device containing a fat filesystem.
much bigger than my flash storage or my memory.
So statically creating the file system is out of the question.
>
> What you can't do is modify the contents of that file while it's
> mounted on another machine as a USB mass storage device - even if it's
> mounted read only, the remote machine isn't expecting the contents of
> the disk to go and change unexpectedly, as that could break any
> caching of data or metadata by the remote system.
I'm clear on that.
But if changes happen on the embedded device I still can unload
g_file_storage and start it again
Dan Danillo
No, it must be simple mass storage.
Dan
Dan Danillo
Imagine a mounted SD-Card combined with data from the embedded system.
Representing multiple partitions or devices are no option either.
Dan
Michael Schnell
>
> No, it must be simple mass storage.
>
o some means needs to be introduced that the files don't change while a
connection is established.
-Michael
cs_po...@hotmail.com
> On 18 Jul., 19:17, cs_post...@hotmail.com wrote:> Create a fat filesystem inside an image file on your linux file
> > system. Point gadget at this storage space, and it will represent it
> > to the USB host as a mass storage device containing a fat filesystem.
>
> Good idea, only problem is, that the file system I want to provide is
> much bigger than my flash storage or my memory.
> So statically creating the file system is out of the question.
Ah, okay, point gadget at /dev/zero then...
Seriously, where were you planning to get the data from?
Wherever the data is coming form, if you can represent that as a block
device, you should be able to point gadget at it.
cs_po...@hotmail.com
How about some foolery in the SD drivers to make a local image file
containing the local data overlay some clusters on the SD-card, and
put a dummy file in it's FAT pointing to them?
Actually, it's probably better to write a wrapper translation driver
the wraps the SD block device and does the substitution. But i might
be tempted to try it by modifying the SD driver first...
I think the big problem would be if somebody else wrote to the card
and moved your dummy file - actually, that could be handled just by
finding it and telling the translator where it is. It's if they made
it non-contiguous that you'd be in pain.
Dan Danillo
That was the plan.
sry I'm keept bussy here.
Dan
Dan Danillo
Yep, thats an issue, but first let me ghet started.
Dan
Dan Danillo
Modify an existing driver - don't like that idea.
The data on the sd-card wont change, the data on the fs might change.
If I can notice that I can disconnect unload g_file store and load it
again.
Dan
cs_po...@hotmail.com
> > How about some foolery in the SD drivers to make a local image file
> > containing the local data overlay some clusters on the SD-card, and
> > put a dummy file in it's FAT pointing to them?
> > Actually, it's probably better to write a wrapper translation driver
> Modify an existing driver - don't like that idea.
Then go with the wrapper driver?
> The data on the sd-card wont change, the data on the fs might change.
> If I can notice that I can disconnect unload g_file store and load it
> again.
Not sure the host PC is going to be really happy about you doing
that. It will survive the disconnect, probably at the cost of an
eventual error message to the user, but you may have to re-enumerate
to reconnect. I'd be worried about it not re-discovering you.