File in memory: A useful feature or dead code?

[bel]

For those who embed CivetWeb into their application, CivetWeb offers a feature "file in memory" (users of the pre-built binaries can stop reading here - they are not affected).

You can pass a callback "open_file" to "mg_start", to tell CivetWeb not to read a file from the disk, but take the data from some memory buffer prepared by the embedding application.

    /* Called when civetweb tries to open a file. Used to intercept file open
       calls, and serve file data from memory instead.
       Parameters:
          path:     Full path to the file to open.
          data_len: Placeholder for the file size, if file is served from
                    memory.
       Return value:
         NULL: do not serve file from memory, proceed with normal file open.
         non-NULL: pointer to the file contents in memory. data_len must be
           initialized with the size of the memory block. */
    const char *(*open_file)(const struct mg_connection *,
                             const char *path,
                             size_t *data_len);


Does anyone use this feature?
How are you using it?

From my point of view, this feature is superseded by the "mg_add_request_handler" function added by Thomas in CivetWeb 1.4 (September 2013). Instead of writing a callback returning a pointer for every known path, you call mg_add_request_handler for every known path and call mg_write there. Furthermore, the "file in memory" feature cannot know when the returned memory buffer is in use, and when it can be freed (except after the server exits), so *safely* it can only be used for const data. The handler callback can deliver different data for different users, different calls, ...

The original "file in memory" feature still exists for compatibility reasons, but I don't see any added value.

1. keep it as it is (don't extend, don't drop, just like the last years)
2. drop this feature to simplify the code, and make it slightly smaller and easier to maintain (keep it as "legacy interface" with a proper define for a version or two, provide a "how to replace" instruction and finally delete it
3. extend this feature in some way to add value, for example by allowing to read and write .htpasswd files - but I think this is nothing a ram-disk would not do as well
4. any other ideas?
   

[Jeroen Walter]

I vote for dropping it.

I've never used it, mainly because it is only safe for const data and in that case you can most likely use a static file anyway or use mg_set_request_handler with a pattern if you don't want to add a handler for every path.

[Hansi PIERRE]

Yep ... I'd drop it too. I don't use it, and i don't think i will.

--
Sourceforge
https://sourceforge.net/projects/civetweb/
---
You received this message because you are subscribed to the Google Groups "civetweb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to civetweb+unsubscribe@googlegroups.com.
To post to this group, send email to cive...@googlegroups.com.
Visit this group at https://groups.google.com/group/civetweb.
To view this discussion on the web visit https://groups.google.com/d/msgid/civetweb/97de7061-e699-4002-b115-8d6a45aae19c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Joe Mucchiello]

I can jump on board the drop it bandwagon. It isn't needed any more. Mark it as deprecated in 1.9 if you can,

[bel]

Since no one seems need this feature, I disabled now, but left a "MG_USE_OPEN_FILE" define to reactivate it.
In case anyone needs this feature, please drop a comment here within the next 3 months.
Maybe there is a use case, we just do not see in the moment.

[snakat]

I'm still use it for android app, in case of reading the file in the 'asset' directory.

[bel]

On Thursday, March 30, 2017 at 10:34:17 AM UTC+2, snakat wrote:

I'm still use it for android app, in case of reading the file in the 'asset' directory.

I understand it is still used - it has been there for longer than the mg_set_request_handler interface I would recommend for new code. It would be very important for us to understand if there is any use case that *due to the different principle* cannot be solved with the new interface. If there is no such case, we can think of a way to slowly discontinue it (mark as deprecated, provide instructions to port the existing code, set #ifdef MG_LEGACY_INTERFACE, keep it for a release or two, and finally remove it). This would be a 1-2 year process.
But I would not do it, if we already know there is something that could not be replaced with reasonable effort.

I would like to discuss a replacement scenario on a real example, if you are open for this.

[Joe Mucchiello]

My issue with the current implementation is memory ownership. There's no way to find out when the civetweb library is finished with the file. And that's why I've avoided the interface.

I can easily see it being replaced with the new request handler interface. Back when there was just "begin request", it made sense to have a shortcut file handler.

[bel]

It seems no one will really miss this feature. Some applications that currently use it can be adapted (I will give some brief instructions).
But finally the code will be simpler and cleaner, if we drop this feature after the upcoming 1.10 release.

(Upcoming = after some summer vacation)

[Joe Mucchiello]

I found a use for it. Templates:

You put a file in the normal directory, like index.html. And in the file you put codes like {{USERID}}. Then you use this interface to replace the variables with proper text. It makes maintenance of the files so much easier.

Kind of a poor man's PHP.

Still, the issue remains "how to deal with freeing the memory".

But I did find the open file interface much cleaner than the equivalent request handler interface since the files are just sitting there in the web root instead of being virtual and tucked away in a template directory.

But get rid of it, it's an edge case only because I'm not using LUA at all.