How to implement per file command line arguments?

[TheDeviloper]

Hello,

I am currently in the process of fixing and extending the Premake Ninja generator to fit our needs.
An important requirement is the support of per file options. This is requried to support other instruction sets for some files (SSSE3, AVX and so on).
I have implemented the Ninja part of the change but I can't find a good way to merge the file configuration with the project configuration to determine the total command line arguments for a given fileconfig object. Just concatenating both command line argument strings obviously does not work, since some switches can be contradicting.

The current code for determining the command line arguments looks like the following.
I basicly took the original Premake Ninja code and turned the individual parts of the command line into functions that can take filecfg object too, but now they don't return the complete commandline. (Instead they return only the changed part like '-mssse3')
    local cc = ""
    local cxx = ""
    local ar = ""
    local link = ""
   
    if toolset_name == "msc" then
        -- TODO premake doesn't set tools names for msc, do we want to fix it ?
        cc = "cl"
        cxx = "cl"
        ar = "lib"
        link = "cl"
    else
        if (toolset_name == "gcc") and (not cfg.gccprefix) then cfg.gccprefix = "" end
        cc = toolset.gettoolname(cfg, "cc")
        cxx = toolset.gettoolname(cfg, "cxx")
        ar = toolset.gettoolname(cfg, "ar")
        link = toolset.gettoolname(cfg, iif(cfg.language == "C", "cc", "cxx"))
    end

    ---------------------------------------------------- figure out settings
    local globalincludes = {}
    table.foreachi(cfg.includedirs, function(v)
        -- TODO this is a bit obscure and currently I have no idea why exactly it's working
        globalincludes[#globalincludes + 1] = project.getrelative(cfg.workspace, v)
    end)
   
    local buildopt = function(cfg) return ninja.list(cfg.buildoptions) end
    local cflags = function(cfg) return ninja.list(toolset.getcflags(cfg)) end
    local cppflags = function(cfg) return ninja.list(toolset.getcppflags(cfg)) end
    local cxxflags = function(cfg) return ninja.list(toolset.getcxxflags(cfg)) end
    local warnings = function(cfg) if toolset_name == "msc" then return ninja.list(toolset.getwarnings(cfg)) else return "" end end
    local defines = function(cfg) return ninja.list(table.join(toolset.getdefines(cfg.defines), toolset.getundefines(cfg.undefines))) end
    local includes = function(cfg) return ninja.list(toolset.getincludedirs(cfg, globalincludes, cfg.sysincludedirs)) end
    local forceincludes = function(cfg) return ninja.list(toolset.getforceincludes(cfg)) end -- TODO pch
    local ldflags = function(cfg) return ninja.list(table.join(toolset.getLibraryDirectories(cfg), toolset.getldflags(cfg), cfg.linkoptions)) end
    local all_cflags = function(cfg)
            return buildopt(cfg) .. cflags(cfg) .. warnings(cfg) .. defines(cfg) .. includes(cfg) .. forceincludes(cfg)
        end
    local all_cxxflags = function(cfg)
            return buildopt(cfg) .. ninja.remove_identical_words(cflags(cfg), cxxflags(cfg)) .. cppflags(cfg) .. warnings(cfg) .. defines(cfg) .. includes(cfg) .. forceincludes(cfg)
        end

How can I implement per file configuration the simplest way without breaking it if the available options change?
Also I find it a bit weird that seemingly every Premake target has to concatenate the different command line option groups itself. Is there maybe a better way of doing this all together?

[starkos]

Per-file settings are definitely the weakest link right now. One part of the problem is the ability to query the settings for a specific file, without also bringing in everything from the project and the configuration level. I'm working on something for this right now (really, I am), and hopefully will having something to show for it soon (in open source time). The other part of it is the ability to turn those settings into the right switches for the toolset. I don't think anyone is working on that part right now. In either case, suggestions for improvements are always welcomed.

[TheDeviloper]

If I understand you correctly, there is no solution right now.
Can you elaborate a bit on how you want to add it and what timespan you have in mind when you talk about 'open source time'?

I dont know the inner workings of Premake very well, but the functionalities I am looking for are essentionally these two:

• A simple 'merge' function that combines different setting objects (One being dominant by overwritting the base setting if both exist)
• And a 'buildCommandLine' function of the toolset that takes a setting objects and returns a complete and usable command line.
  

[starkos]

"Open source time" means that it will get done when I can get it done, subject to my other priorities (i.e. my day job).

[TheDeviloper]

After doing a lot of reverse engineering and I found a way that seems to work.
In case other people run into that problem, here is my solution:

function ninja.mergeCfgs(base, add)
    local new_cfg = p.context.extent(base); -- copy the configuration
 
    -- merge all values present in 'add'
    for index, field in pairs(p.field._list) do -- Iterating only over the relevant entries in 'add' does not
        -- always work. It seems like generated tables do not properly work with for each loops.
        local add_value = add[field.name]
        if not ((add_value == nil) or ((type(add_value) == "table") and ninja.tableIsEmpty(add_value))) then -- ignore empty entries...
            local new_field = p.field.merge(field, base[field.name], add_value)
            new_cfg[field.name] = new_field
        end
    end
    return new_cfg
end