Bleve indexes in a read-only folder

[Chris Kollett]

I am attempting to use Bleve in an environment where the directory containing the search indexes is read only. I assumed that it would be straightforward to use Bleve in this configuration simply by opening the index in read-only mode:

index, err := bleve.OpenUsing(pathToIndex, map[string]interface{}{"read_only": true})

However, it appears that this is not currently supported; I encounter an "Access is denied" error attempting to open the root.bolt file. It appears that this is actually due to the default behavior of Bolt. In db.go I see:

    flag := os.O_RDWR

    if options.ReadOnly {

        flag = os.O_RDONLY

        db.readOnly = true

    }

    db.openFile = options.OpenFile

    if db.openFile == nil {

        db.openFile = os.OpenFile

    }

    // Open data file and separate sync handler for metadata writes.

    var err error

    if db.file, err = db.openFile(path, flag|os.O_CREATE, mode); err != nil {

        _ = db.close()

        return nil, err

    }

The code that I've highlighted seems to indicate that, even when you specify the os.O_RDONLY flag, Bolt adds the os.O_CREATE flag as well, and as a result I get an error.

I realize that this is actually a limitation of Bolt, and not Bleve, but it seems like Bleve should be able to support this scenario. Given that Bolt allows the calling code to provide its own openFile function, perhaps it would be possible to add a workaround to Bleve that would allow Bolt to open an existing root.bolt file in a read-only folder.

Has anyone else encountered this issue and successfully worked around it?

Thank you,

Chris

[Chris Kollett]

I think a potential fix for this could be made in scorch.go. Bleve currently does the following when using indexes in read-only mode:

    if s.readOnly {

        rootBoltOpt.ReadOnly = true

    }

Bolt allows you to specify your own function for opening files, however, so Bleve could be updated to add a function that looks something like:

    func openReadOnlyBoltFile(path string, flag int, mode os.FileMode) (*os.File, error) {

    file, err := os.OpenFile(path, flag, mode)

    if err != nil {

        file, err = os.OpenFile(path, os.O_RDONLY, mode)

    }

    return file, err

    }

Then the if block above could be modified to:

    if s.readOnly {

        rootBoltOpt.ReadOnly = true

        rootBoltOpt.OpenFile = openReadOnlyBoltFile

    }

This resolves the issue for me, but admittedly looks more like a workaround than a clean fix. I'd welcome any thoughts on whether this looks like a viable approach for this issue.

Thank you,

Chris

[Abhinav Dangeti]

Hello @ckollet, apologies for not getting back to you on this sooner.

I see that you've created an issue for this here - https://github.com/blevesearch/bleve/issues/1623.

Let's move this conversation to the github ticket.