security

[Lukas Kahwe Smith]

Aloha,

I would like to bring up another topic:
collaboration on security

One area here specifically I am thinking of here are policies for how to deal with security issues. This here is for example the Symfony2 process:
http://symfony.com/doc/current/contributing/code/security.html

On top of that I would also like to bring up the security advisories database:
http://fabien.potencier.org/article/74/the-php-security-advisories-database

Note: I am not an expert in this area and there seems to be an obvious issue that the above mentioned database needs additional steps to get CVE's added. Not sure if this processes could be made smoother (or even semi redundant) with some additional work.

Given that more and more members (and the community in general) is adopting Composer it would be awesome if one could check for known security vulnerabilities for any PHP project using tools like the sensio security checker:
https://security.sensiolabs.org/check

One additional challenge, which I am not sure if they have been addressed yet, is that Composer tends to be a developer tool used on the CLI. Now many applications also allow users to install additional code via an admin UI. I know that Nils originally started Composer for this exact use case in phpBB, but I am not sure what the state is here .. ie. if such applications are leveraging Composer when they install dependencies (extensions/modules/plugins/Bundles) via an admin UI.

regards,
Lukas

[Larry Garfield]

I am unclear if you are proposing a common "best practice" document or a
common reporting and notification tool. There was discussion a while
ago about a common reporting and notification mechanism, ie, some
standard machine-digestable and human-readable format to make it easier
to distribute security advisories to users. I fully support the latter
idea, but have been occupied with numerous other matters so haven't
pursued it myself.

--Larry Garfield

[Lukas Smith]

> On 30 Oct 2014, at 07:00, Larry Garfield <la...@garfieldtech.com> wrote:
>
>> On 10/30/2014 12:48 AM, Lukas Kahwe Smith wrote:
>> Aloha,
>>
>> I would like to bring up another topic:
>> collaboration on security
>>
>> One area here specifically I am thinking of here are policies for how to deal with security issues. This here is for example the Symfony2 process:
>> http://symfony.com/doc/current/contributing/code/security.html
>>
>> On top of that I would also like to bring up the security advisories database:
>> http://fabien.potencier.org/article/74/the-php-security-advisories-database
>>
>> Note: I am not an expert in this area and there seems to be an obvious issue that the above mentioned database needs additional steps to get CVE's added. Not sure if this processes could be made smoother (or even semi redundant) with some additional work.
>>
>> Given that more and more members (and the community in general) is adopting Composer it would be awesome if one could check for known security vulnerabilities for any PHP project using tools like the sensio security checker:
>> https://security.sensiolabs.org/check
>>
>> One additional challenge, which I am not sure if they have been addressed yet, is that Composer tends to be a developer tool used on the CLI. Now many applications also allow users to install additional code via an admin UI. I know that Nils originally started Composer for this exact use case in phpBB, but I am not sure what the state is here .. ie. if such applications are leveraging Composer when they install dependencies (extensions/modules/plugins/Bundles) via an admin UI.
>>
>> regards,
>> Lukas
>
> I am unclear if you are proposing a common "best practice" document or a common reporting and notification tool. There was

I am proposing both. ie. each of us sharing their current process to see if we can learn from each other. One interesting topic here is how to deal with key up stream projects.

Additionally maybe we can also standardize how we publish security infos.

> discussion a while ago about a common reporting and notification mechanism, ie, some standard machine-digestable and human-readable format to make it easier to distribute security advisories to users. I fully support the latter idea, but have been occupied with numerous other matters so haven't pursued it myself.
>
> --Larry Garfield
>

> --
> You received this message because you are subscribed to the Google Groups "PHP Framework Interoperability Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+u...@googlegroups.com.
> To post to this group, send email to php...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/5451D403.1010809%40garfieldtech.com.
> For more options, visit https://groups.google.com/d/optout.

[Pádraic Brady]

On 30 October 2014 05:48, Lukas Kahwe Smith <sm...@pooteeweet.org> wrote:

Aloha,

I would like to bring up another topic:
collaboration on security

One area here specifically I am thinking of here are policies for how to deal with security issues. This here is for example the Symfony2 process:
http://symfony.com/doc/current/contributing/code/security.html

On top of that I would also like to bring up the security advisories database:
http://fabien.potencier.org/article/74/the-php-security-advisories-database

It should be fairly simple to agree on a security policy document if we can agree its scope, and by that I mean defining the responsibilities of all parties to a security report to a sufficient degree as to eliminate surprises/doubt. A problem at the moment is that policies have an extremely narrow scope that sets out how a project will deal with an issue without reference (broadly – it differs by project) to factors like a timeline, deadline, peer review, responsibilities of the reporter, responsibilities of the project, a contact email, support (full/part/none?) for responsible disclosure practices, cough…limited liability for reporters…cough,  etc.

Many current policies, within PHP’s scope, can really be summarised as: report it, please don’t disclose it, and we’ll fix it. Even the one page policies can be reduced to this in many cases.

Reporters however can have a whole other set of concerns that policies almost never address which can lead to friction (some good, some clearly not). Some will ask for your going “compensation rate”, dictate disclosure times due to conference/publication deadlines of their own, harass you by email every few days (we consider it good manners to get an update regularly ;)), won’t talk about practical examples in case you sue them for pasting stuff in a search box, may simply lack the experience/time/funds to offer a practical example for an “obvious” vulnerability, and so on. Bear in mind some reporters are making reports as part of their job, not simply out of work hours as a social service.

In recent years, the corporate world has slowly started to come around to the idea of being reporter-friendly, offering bounties, no-liability clauses, stiffer timelines for fixes, more visible credit on their websites (e.g. Google’s Hall of Fame), etc.

On exchanging security information, we looked into this before and we briefly looked at some existing formats. There are a few out there, but they tend towards the cumbersome end of the scale using XML or unspecified JSON ports. For the most part, all we need are:

1.    Project ID
2.    Version IDs
3.    Vulnerability IDs (>1 to allow for both local and global IDs like CVE)
4.    Short Description
5.    URL (possibly optional)

Beyond that you get into information that, while potentially useful, starts complicating the beast or becomes subjective data like severity ratings. This could be added as optional, if truly needed by anyone, without altering the core compulsory points.

You also then need a service to collect all of this data, turn it into something useful, and then distribute it back to project users. There’s also the problem of getting the PHP community sufficiently motivated to actually start reporting this data accurately. It remains a fairly common bad habit to either not disclose security fixes or disclose them in an inaccurate fashion so this would not be a fully reliable system though just getting our own membership online would be nice.

Paddy

--
Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
Zend Framework Community Review Team
Zend Framework PHP-FIG Representative

[Lukas Kahwe Smith]

> On 30 Oct 2014, at 13:06, Pádraic Brady <padrai...@gmail.com> wrote:

> You also then need a service to collect all of this data, turn it into something useful, and then distribute it back to project users. There’s also the problem of getting the PHP community sufficiently motivated to actually start reporting this data accurately. It remains a fairly common bad habit to either not disclose security fixes or disclose them in an inaccurate fashion so this would not be a fully reliable system though just getting our own membership online would be nice.

I realize the track record is not so great but we are all here to improve things :)
In this spirit we could have a PSR that defines a security process. One of the key things I hope is that this will lead to a more structured way these issues are reported. Which then means its easier for people to do automated checks on their apps.

regards,
Lukas

[Michael Cullum]

Regarding the latter part of your message, I am currently working on a UI which allows composer management from an admin UI and I intend to make a lot of the code re-usable. Saying this, this is rather a separate issue/discussion from the one in regards to security?

On your primary point of security, I too am slightly unsure of what your actual proposal is beyond an initiative to get FIG members to all centralise their security vulnerabilities in one place (the FriendsOfPHP Security Advisories Repo).

I cannot really see the usefulness of an agreement of how to handle security vulnerabilities except maybe as something new OS projects can look to (Best Practices Recommendation) when considering how they should handle security vulnerabilities as existing projects all have their well established ways of handling security issues which differ in some areas (Release ASAP after identifying issue and patching vs Release Windows [Similar to Microsoft's Patch Tuesday]) but generally all agree on the good core foundations which Paddy outlined.

I do agree however that a centralised way of handing security advisories is a good idea but this already exists in the form of the Security Advisories Repo, I think the mission should simply be to get more people to make use of it?

--
Thanks,

Michael C
phpBB

--
You received this message because you are subscribed to the Google Groups "PHP Framework Interoperability Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+u...@googlegroups.com.
To post to this group, send email to php...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/04AC1CFA-00D3-4FBA-8D34-F97AE1000713%40pooteeweet.org.

[Lukas Smith]

> On 30 Oct 2014, at 16:11, "Michael Cullum" <m...@michaelcullum.com> wrote:
>
> Regarding the latter part of your message, I am currently working on a UI which allows composer management from an admin UI and I intend to make a lot of the code re-usable. Saying this, this is rather a separate issue/discussion from the one in regards to security?

yeah its separate but related in that many of us are pushing composer as the tool of choice for dependencies and so it could be out single source for everything installed to compare against a central database of security issues.

> On your primary point of security, I too am slightly unsure of what your actual proposal is beyond an initiative to get FIG members to all centralise their security vulnerabilities in one place (the FriendsOfPHP Security Advisories Repo).
>
> I cannot really see the usefulness of an agreement of how to handle security vulnerabilities except maybe as something new OS projects can look to (Best Practices Recommendation) when considering how they should handle security vulnerabilities as existing projects all have their well established ways of handling security issues which differ in some areas (Release ASAP after identifying issue and patching vs Release Windows [Similar to Microsoft's Patch Tuesday]) but generally all agree on the good core foundations which Paddy outlined.

well one aspect is learning from each other. the other by having some level of least common denominator standard we ease the hurdle for security researchers due submit issues in a way that fits the process of the projects.

> I do agree however that a centralised way of handing security advisories is a good idea but this already exists in the form of the Security Advisories Repo, I think the mission should simply be to get more people to make use of it?

indeed. in order to start small, that seems like the ideal low hanging fruit to focus on

regards,
Lukas

[Pádraic Brady]

On 30 October 2014 15:11, Michael Cullum <m...@michaelcullum.com> wrote:

Regarding the latter part of your message, I am currently working on a UI which allows composer management from an admin UI and I intend to make a lot of the code re-usable. Saying this, this is rather a separate issue/discussion from the one in regards to security?

Very separate. I think the actual implementation details can lean on whatever PSRs might come about from this, but implementation is largely beyond the remit of this group. Of course, as individuals, we can actively look into implementation and see if Composer would consider adopting an implementation. Any PSRs needn't be contingent on having Composer or any other possible implementation.
 

On your primary point of security, I too am slightly unsure of what your actual proposal is beyond an initiative to get FIG members to all centralise their security vulnerabilities in one place (the FriendsOfPHP Security Advisories Repo).

Being practical, we have four objectives of sorts:

1. Agreeing on a single baseline security policy that member projects could adopt or claim compatability with.
2. Agreeing a set of additional recommended but optional security policy claims. Sort of a checklist that can be published showing quickly what a project does or does include in addition to the baseline policy.
3. Agreeing a format for exchanging security vulnerability data.
4. Agreeing how to discover the URL for such data for automated collection.

None of these are that difficult really, though I'm sure we'll find points to debate :). That should be enough to account for say, two PSRs (policy + data format/discovery), which can enable an implementation to forge ahead.

On one specific, I don't think establishing a central repository should be the focus. Projects can put the data somewhere local and discoverable (e.g. their git repo) and any implementation can then build its own central data store from scratch quite easily.
 

I cannot really see the usefulness of an agreement of how to handle security vulnerabilities except maybe as something new OS projects can look to (Best Practices Recommendation) when considering how they should handle security vulnerabilities as existing projects all have their well established ways of handling security issues which differ in some areas (Release ASAP after identifying issue and patching vs Release Windows [Similar to Microsoft's Patch Tuesday]) but generally all agree on the good core foundations which Paddy outlined.

Given PHP-FIG literally has interoperability in its name... Most of us have ad-hoc disclosure methods which aren't compatible. Just looking at current members, we don't all assign IDs (homegrown or CVEs). We don't all even publish advisories in the traditional sense. Some of us may just post a blog post, some may have dedicated RSS or Atom feeds, some will just use commit logs. Interoperability in this regard would be a relatively tiny step for all members.
 

I do agree however that a centralised way of handing security advisories is a good idea but this already exists in the form of the Security Advisories Repo, I think the mission should simply be to get more people to make use of it?

Put simply, given the following options, which would be preferable?

1. Contribute data to a central repository that you do not control by opening a pull request and awaiting peer review of the request to ensure it's valid; or
2. Commit a new xxx.json version to your local repository.

The second is simpler and more convenient. It also implicitly leaves a project with control over its vulnerability data should anyone ever decide to verify centralised data from another provider to spot forgeries or false positives.

[Larry Garfield]

On 10/30/14, 11:33 AM, Pádraic Brady wrote:

> Being practical, we have four objectives of sorts:
>
> 1. Agreeing on a single baseline security policy that member projects
> could adopt or claim compatability with.
> 2. Agreeing a set of additional recommended but optional security policy
> claims. Sort of a checklist that can be published showing quickly what a
> project does or does include in addition to the baseline policy.
> 3. Agreeing a format for exchanging security vulnerability data.
> 4. Agreeing how to discover the URL for such data for automated collection.
>
> None of these are that difficult really, though I'm sure we'll find
> points to debate :). That should be enough to account for say, two PSRs
> (policy + data format/discovery), which can enable an implementation to
> forge ahead.
>
> On one specific, I don't think establishing a central repository should
> be the focus. Projects can put the data somewhere local and discoverable
> (e.g. their git repo) and any implementation can then build its own
> central data store from scratch quite easily.

> Given PHP-FIG literally has interoperability in its name... Most of us
> have ad-hoc disclosure methods which aren't compatible. Just looking at
> current members, we don't all assign IDs (homegrown or CVEs). We don't
> all even publish advisories in the traditional sense. Some of us may
> just post a blog post, some may have dedicated RSS or Atom feeds, some
> will just use commit logs. Interoperability in this regard would be a
> relatively tiny step for all members.

> Put simply, given the following options, which would be preferable?
>
> 1. Contribute data to a central repository that you do not control by
> opening a pull request and awaiting peer review of the request to ensure
> it's valid; or
> 2. Commit a new xxx.json version to your local repository.
>
> The second is simpler and more convenient. It also implicitly leaves a
> project with control over its vulnerability data should anyone ever
> decide to verify centralised data from another provider to spot
> forgeries or false positives.
>
> Paddy

When we last discussed this, my thinking was that we should establish a
feed-capable format for recording SAs, and then projects should
"conspicuously publish" a URL where SAs can be retrieved from. For
smaller projects that would likely be just a file in their GitHub repo,
or some 3rd party aggregator that will host it for them. For larger
projects or meta projects (eg, Drupal) that would have a larger number
of SAs because they cover many projects, they could have an actual feed.
In either case, someone could poll the URL and check their own
project's dependencies, or an aggregator like Sensio could do so, or
whatever. Let the market sort that part out, as long as it's just a URL
that returns some known format our job is done.

In practice I figure "conspicuously publish" would probably turn into a
key in composer.json, but that's not something we'd require.

The advantage of a feed-capable format is push-notifications. It would
be great if we could support some sort of pubsub mechanism for
aggregators. It's an optional piece, but could be useful to avoid
pinging the world to death. :-)

I don't think a central authoritative repository of SAs is wise, if for
no other reason then "who the hell is going to maintain that?" I am not
volunteering. :-)

--Larry Garfield

[Lukas Kahwe Smith]

Yeah, a standardized format along with an Atom/RSS feed sounds indeed like a good plan. Then people can create harvesters as needed.

We could use the FriendsOfPHP YAML based format as inspiration on what kind of data would be needed:
https://github.com/FriendsOfPHP/security-advisories/blob/master/symfony/dependency-injection/2012-08-28.yaml

regards,
Lukas

[Timo H]

Glad to see this kind of effort is being worked on.

A few quick comments:

I think we should aim for local and centralized data. I.e. every participating project uses their own local repository (for example xxx.json as suggested by Pádraic), but they can also add itself's to the central repository which collects security data from all projects that are participating (via the standard format and projects' url information).

Maybe it would be worth to discuss if besides the list Pádraic suggested, we could collect also information about vulnerability types (something for example like http://www.cvedetails.com/vendor/2105/Moodle.html). Drawback is that the individual project's effort is greater if more info is needed to publish a security alert (maybe make it optional). But this data could come useful i.e. when browsing the central repository.

About reporting security bugs, I think we should indeed aim to be as "reporter friendly" as possible. Anyway possibly one of the biggest cause of friction between reporters and projects is "silence of the project maintainers". This can be hard to "standardise", i.e. by requiring that a project must reply in certain amount of time, but maybe it is better to fix by "education":

Try to respond even something to the reporter, probably anything is better than completely radio silence.

etc.

Timo

[Michael Cullum]

So the fruits of this would be:

1) A recommendation on how to issue advisories, handle security reports and responsible disclosure

2) Everyone has a file in which their security advisories are listed which can then be aggregated together application side by apps such as the SL Security Checker or put their SA file in a centralised repository instead of their own such as the FriendsOfPHP one. Perhaps even build into composer itself something which checks the security of existing versions and having the location of these SA files defined in composer.json files.

This sounds like a good idea about an important issue, especially regarding point 2. What sort of format do we want to have this in? A traditional PSR seems a little out of place in this situation but then there is no formal definition of what constitutes a PSR (We have [3] interfaces, [4] standards and a guide). Point 1 sounds something like a guide whereas Point 2 is a standard.

I’d be interested to see what the folks from Composer have to say in regards to point 2.

--

Thanks,

Michael C

[Larry Garfield]

On 10/30/14, 4:19 PM, Michael Cullum wrote:
> So the fruits of this would be:
> 1) A recommendation on how to issue advisories, handle security reports
> and responsible disclosure
> 2) Everyone has a file in which their security advisories are listed
> which can then be aggregated together application side by apps such as
> the SL Security Checker or put their SA file in a centralised repository
> instead of their own such as the FriendsOfPHP one. Perhaps even build
> into composer itself something which checks the security of existing
> versions and having the location of these SA files defined in
> composer.json files.

s/file/URL/ where the SA lives, but otherwise yes.

> This sounds like a good idea about an important issue, especially
> regarding point 2. What sort of format do we want to have this in? A
> traditional PSR seems a little out of place in this situation but then
> there is no formal definition of what constitutes a PSR (We have [3]
> interfaces, [4] standards and a guide). Point 1 sounds something like a
> guide whereas Point 2 is a standard.

We don't technically have a mechanism for publishing "guidelines", just
standards. I don't know how we'd address part 1 this way. For part 2,
it would be some file format PSR specification, possibly with external
references. (Eg, "the JSON format below, wrapped up in Atom XML, use
pubsubhubbub if you want", or something like that.)

> I’d be interested to see what the folks from Composer have to say in
> regards to point 2.

--Larry Garfield

[Pádraic Brady]

Hi Larry,

On 30 October 2014 18:25, Larry Garfield <la...@garfieldtech.com> wrote:

When we last discussed this, my thinking was that we should establish a feed-capable format for recording SAs, and then projects should "conspicuously publish" a URL where SAs can be retrieved from.  For smaller projects that would likely be just a file in their GitHub repo, or some 3rd party aggregator that will host it for them.  For larger projects or meta projects (eg, Drupal) that would have a larger number of SAs because they cover many projects, they could have an actual feed.  In either case, someone could poll the URL and check their own project's dependencies, or an aggregator like Sensio could do so, or whatever.  Let the market sort that part out, as long as it's just a URL that returns some known format our job is done.

Could be done either by reusing existing XML elements in RSS & Atom (if anything relevant or reusable) or perhaps more cleanly by defining a separate namespace similar to Dublin Core (DC) that can decorate existing feeds. The main complication thereafter would be discovery assuming we want to support automated trawling from some base URL (e.g. a git repo URI). Yes, we can push discovery out to some theoretical central service, but that's a bit like putting all the eggs in one basket. I'd like it be fairly open without a strict registration requirement to figure out where the data is - let others enter the playpen as freely as possible.

 

In practice I figure "conspicuously publish" would probably turn into a key in composer.json, but that's not something we'd require.

Again, not much of a fan of relying on any one particular implementation, in this case Packagist. If Composer goes kaput tomorrow, would be nice if the whole security reporting architecture did not collapse alongside it, but can be rebuilt by other services with a minimum of fuss.

 

The advantage of a feed-capable format is push-notifications.  It would be great if we could support some sort of pubsub mechanism for aggregators.  It's an optional piece, but could be useful to avoid pinging the world to death. :-)

Whatever RSS/Atom are currently compatible with, e.g. pubsubhubbub or similar.

 

I don't think a central authoritative repository of SAs is wise, if for no other reason then "who the hell is going to maintain that?"  I am not volunteering. :-)

There's no need to assume a specific central authority in any eventual specification. Whatever is arrived at may support any number of "consumers". Sure that sounds similar, but the assumptions around linking this to Composer explicitly or assuming Packagist will get involved are by no means certain, nor should they be assumed to be permanent and required building blocks. Any eventual tool could just query Packagist, as an example, or just use the project data and version info to discover the data, read it, and do the cross-referencing locally. No central intermediary strictly required...

[Larry Garfield]

On 10/30/14, 4:48 PM, Pádraic Brady wrote:
> Hi Larry,
>
> On 30 October 2014 18:25, Larry Garfield <la...@garfieldtech.com

I think we're saying the same thing. We just say "make this URL
available in some way". Whatever centralization and aggregation that
then happens, happens. If that's Packagist, cool. If it's not
Packagist, also cool. Sensio or someone like that could encourage
people to just register their known URL with them and take it from
there. That's out of our hands. We just say "provide a URL where one
can get X data".

--Larry Garfield

[Pádraic Brady]

On 30 October 2014 22:04, Larry Garfield <la...@garfieldtech.com> wrote:

I think we're saying the same thing.  We just say "make this URL available in some way".  Whatever centralization and aggregation that then happens, happens.  If that's Packagist, cool.  If it's not Packagist, also cool.  Sensio or someone like that could encourage people to just register their known URL with them and take it from there.  That's out of our hands.  We just say "provide a URL where one can get X data".

Yep, we're on same page. Only clarification would be what I call discovery which is just addressing how to locate the data based on something simpler, such as the repo URI or main website URL. For repos, it would be likely just be by convention - have an xxx.json or xxx.yaml file. For websites, it may need a meta tag indicating the location of the relevant feed. Mostly this is just help reduce manual intervention by having to take to google or a site search. I could also be going overboard - it's been known to happen ;).

[Timo H]

perjantai, 31. lokakuuta 2014 0.23.25 UTC+2 Pádraic Brady kirjoitti:

On 30 October 2014 22:04, Larry Garfield <la...@garfieldtech.com> wrote:

I think we're saying the same thing.  We just say "make this URL available in some way".  Whatever centralization and aggregation that then happens, happens.  If that's Packagist, cool.  If it's not Packagist, also cool.  Sensio or someone like that could encourage people to just register their known URL with them and take it from there.  That's out of our hands.  We just say "provide a URL where one can get X data".

Yep, we're on same page. Only clarification would be what I call discovery which is just addressing how to locate the data based on something simpler, such as the repo URI or main website URL. For repos, it would be likely just be by convention - have an xxx.json or xxx.yaml file. For websites, it may need a meta tag indicating the location of the relevant feed. Mostly this is just help reduce manual intervention by having to take to google or a site search. I could also be going overboard - it's been known to happen ;).

 Wouldn't we need the central repository to be something where participating projects can "register" their project, so that the central repository doesn't have to do any discovery (which I think could easily be a dead end)?

Some kind of moderation is also needed (in case someone wants harass the service by submitting bogus details etc., it could be just removed from the central repository).

I'm thinking the "central repository" to be like a web page which displays participating projects' security alerts, possibly with some nice decorations like bug type grouping etc. But all the actual information is coming from the participating projects' own repositories. And maybe the central repository would share details of all the participating projects, so another entity could create same kind of "central repository service". But anyway, I think we need one "central place" where participating projects can register, so they all become "indexed". Does this sound reasonable?

Timo

[Pádraic Brady]

On 31 October 2014 08:17, Timo H <tim...@gmail.com> wrote:

 Wouldn't we need the central repository to be something where participating projects can "register" their project, so that the central repository doesn't have to do any discovery (which I think could easily be a dead end)?

Some kind of moderation is also needed (in case someone wants harass the service by submitting bogus details etc., it could be just removed from the central repository).

The problem with any central repository is that it would also become a moderator of data. It’s one of the obvious problems with Fabien’s current effort that all vulnerabilities need to be centralised, which means that they need to be moderated. It can never become a free for all resource due to the risk of the incoming reports exceeding the capacity of its moderators (i.e. those with commit authority) to peer review them for accuracy prior to merging.

A decentralised system exports the responsibility back to where it belongs, the projects themselves. The only way to fake/game the system, would be to do something at the project level where it would be self-limited by the project ID (readily verifiable with reference to the git repo URL either locally or via a third party like Packagist).

That doesn’t preclude the existence of central repositories – it just renders them non-essential. They can still play a role documenting vulnerabilities for projects which haven’t opted to use anything suggested here, where moderation and third party tracking DOES then make sense, and where downstream clients actually consuming such data can opt in or out of trusting them as a centralised source. That’s with the implied limitation that data from a specific project will take precedence over data from a centralised source by default (perhaps provide a warning should data conflict so that users can dig deeper and switch trust to the centralised source if its more reliable for a project).

The question however is, do we really need to discuss that in terms of PSRs?
 

I'm thinking the "central repository" to be like a web page which displays participating projects' security alerts, possibly with some nice decorations like bug type grouping etc. But all the actual information is coming from the participating projects' own repositories. And maybe the central repository would share details of all the participating projects, so another entity could create same kind of "central repository service". But anyway, I think we need one "central place" where participating projects can register, so they all become "indexed". Does this sound reasonable?

In a way, we already have a significant number of projects out there pre-indexed by Packagist. Without even implying Packagist have anything to do with security issues, that index can be reused by anyone for other purposes. Before my extended break over the Summer, I was using it myself to conduct a survey of code standards adherence. Note to self: need to finish that...

Given an already public index of project repositories/websites, all you need on top of that is "discovery", a simple set of rules that let you locate the security data based on something like the repo URI.

For projects not registered on Packagist, discovery rules can still apply to however you are including them, e.g. via Composer. If you're not using Composer... We'd need to see specific examples here to identify a treatment.

Paddy

[Timo H]

perjantai, 31. lokakuuta 2014 12.21.04 UTC+2 Pádraic Brady kirjoitti:

On 31 October 2014 08:17, Timo H <tim...@gmail.com> wrote:

 Wouldn't we need the central repository to be something where participating projects can "register" their project, so that the central repository doesn't have to do any discovery (which I think could easily be a dead end)?

Some kind of moderation is also needed (in case someone wants harass the service by submitting bogus details etc., it could be just removed from the central repository).

The problem with any central repository is that it would also become a moderator of data. It’s one of the obvious problems with Fabien’s current effort that all vulnerabilities need to be centralised, which means that they need to be moderated. It can never become a free for all resource due to the risk of the incoming reports exceeding the capacity of its moderators (i.e. those with commit authority) to peer review them for accuracy prior to merging.

I was thinking no project must be part of the centralization, but they can be if they choose to do so (and we should encourage it). And also the reports come "as is" from the joined in projects (no need to moderate them by the "central repository"). The central repositoy is just a central place where the information about participating projects is being stored (no need for possibly complicated auto discovery). This would make it easy to browse/display and keep track of all of the projects and security advisories in one central place (for projects who wants to participate).

(But of course one can find the advisories directly from the project's repository too)
 

A decentralised system exports the responsibility back to where it belongs, the projects themselves. The only way to fake/game the system, would be to do something at the project level where it would be self-limited by the project ID (readily verifiable with reference to the git repo URL either locally or via a third party like Packagist).

Yep this is indeed what I was thinking too, with the exception that if we choose to maintain this "central repository", there needs to be some moderation (to filter possibly fake accounts who just create and create bogus advisories spamming the central repository, and thus decrease the visibility of real advisories).

 

That doesn’t preclude the existence of central repositories – it just renders them non-essential. They can still play a role documenting vulnerabilities for projects which haven’t opted to use anything suggested here, where moderation and third party tracking DOES then make sense, and where downstream clients actually consuming such data can opt in or out of trusting them as a centralised source. That’s with the implied limitation that data from a specific project will take precedence over data from a centralised source by default (perhaps provide a warning should data conflict so that users can dig deeper and switch trust to the centralised source if its more reliable for a project).

Good points, I'd just like to emphasise that if there is one "standard central repository" which collects participating projects, we can easily have "all inclusive" central place for security advisories (showing up in one same place while coming form the projects' itselfs).

 

I'm thinking the "central repository" to be like a web page which displays participating projects' security alerts, possibly with some nice decorations like bug type grouping etc. But all the actual information is coming from the participating projects' own repositories. And maybe the central repository would share details of all the participating projects, so another entity could create same kind of "central repository service". But anyway, I think we need one "central place" where participating projects can register, so they all become "indexed". Does this sound reasonable?

In a way, we already have a significant number of projects out there pre-indexed by Packagist. Without even implying Packagist have anything to do with security issues, that index can be reused by anyone for other purposes. Before my extended break over the Summer, I was using it myself to conduct a survey of code standards adherence. Note to self: need to finish that...

Given an already public index of project repositories/websites, all you need on top of that is "discovery", a simple set of rules that let you locate the security data based on something like the repo URI.

For projects not registered on Packagist, discovery rules can still apply to however you are including them, e.g. via Composer. If you're not using Composer... We'd need to see specific examples here to identify a treatment.

This makes sense, but I'd still prefer there was one single place where I can go with my web browser and browse security advisories (I think the auto discovery is not ideal here). Or is it just me who finds sites like http://www.cvedetails.com/ practical? ;)

Timo

[Pádraic Brady]

On 31 October 2014 11:14, Timo H <tim...@gmail.com> wrote:

This makes sense, but I'd still prefer there was one single place where I can go with my web browser and browse security advisories (I think the auto discovery is not ideal here). Or is it just me who finds sites like http://www.cvedetails.com/ practical? ;)

I suppose the point is that the PHP-FIG is not here to implement, but to offer solutions for interoperability that enable interoperable implementations. There may well be some centralised advisory store, but the possibility of its existence, and the shape it takes, will be determined outside of our control.

What is within our control is how the data is presented, how the data can be discovered, and exactly what that data should include. Once it's out there, folk can build anything they want on top of that.

The point of "discovery" is simply to standardise the task of locating the data given that it will be decentralised to individual projects. Packagist requires having a composer.json in a repository, as a simple example. VersionEye can rely on that "discovery" convention to independently crawl Github and numerous other sources to locate projects and use the same data. It's not dependent on Packagist.

Any sort of service as you envision only needs one thing - a set of project URLs. It can pull those from Packagist (if you can backtrack how its API works - not hard), crawl git repositories, accept anonymous URL submissions, etc. The one thing it doesn't strictly need is registration - unless it's offering some service to registrants or is designed as an opt-in only service. If opt-in, it wouldn't survive for long - some smartass will just crawl everything and outdo it easily then ;). Packagist is different because it's opt-in by design and registration gives you some control over a reserved namespace in its system.

[Dracony]

What about integrating these messages with composer.json in a way that when a person does "composer install" of a version that has a known security bug he would get an appropriate warning message?

Just like we can have multiple versions specified in composer.json perhaps we could add an optional parameter to a version definition that would trigger such a message?

[Timo H]

I suppose the point is that the PHP-FIG is not here to implement, but to offer solutions for interoperability that enable interoperable implementations. There may well be some centralised advisory store, but the possibility of its existence, and the shape it takes, will be determined outside of our control.

Aah right, I missed this.
 

What is within our control is how the data is presented, how the data can be discovered, and exactly what that data should include. Once it's out there, folk can build anything they want on top of that.

 
About the included data, in addition to the 5 point list you mentioned, maybe consider an optional "vulnerability type" field (such information could come handy)

Timo

[Alexandru Guzinschi]

On Friday, October 31, 2014 3:43:00 PM UTC+2, Pádraic Brady wrote:

 

Any sort of service as you envision only needs one thing - a set of project URLs. It can pull those from Packagist (if you can backtrack how its API works - not hard), crawl git repositories, accept anonymous URL submissions, etc. 

I have a small question regarding URL submission

How do you make sure that a submitted URL really belongs to that project ? In other words, what will prevent me to submit something like http://example.com/zend-advisory/ and that URL to be taken into consideration in all reports regarding Zend framework ?

[Lukas Kahwe Smith]

well each project usually as a pretty clearly defined domain where they host docs and various other links. 

regards,

Lukas

[Alexandru Guzinschi]

On Saturday, November 1, 2014 10:31:32 AM UTC+2, Lukas Kahwe Smith wrote:

well each project usually as a pretty clearly defined domain where they host docs and various other links. 

I agree, but starting a big list of "legit" URL's I don't think it's very practical. And besides that, maybe some projects will want a separate/shorter address where they will post such things. If everything should work without human intervention, you need a way to make sure that <url> from submission form is legit for <project name>

[Lukas Kahwe Smith]

the can always announce their shorter url on their main domain. that being said, i dont think that such URLs need to be short.

regards,

Lukas

[Alexandru Guzinschi]

On Saturday, November 1, 2014 10:42:29 AM UTC+2, Lukas Kahwe Smith wrote:

the can always announce their shorter url on their main domain. that being said, i dont think that such URLs need to be short.

That doesn't really solves the validation problem but moves it from one domain to other. You still need a way to make sure that "main domain" belongs to <project> and having a big list with known projects and associated URL's or manual intervention it's not feasible.

[Pádraic Brady]

Packagist has no issues associating projects with URIs... This sounds a lot like a problem that doesn't exist.

Paddy

[Alexandru Guzinschi]

On Saturday, November 1, 2014 11:06:49 AM UTC+2, Pádraic Brady wrote:

Packagist has no issues associating projects with URIs... This sounds a lot like a problem that doesn't exist.

On packagist it's not important if a project's homepage is misleading because the project source/canonical matters, which is namespaced so I can't mislead someone to believe that my "zend-framework" package it's the real zend framework. When you submit anonymous URL's you don't have that namespace available and you need a way to validate the vendor behind that URL.

On packagist a vendor submits their package and publish a link to packagist on their website, so anyone who browse their website can see those links. When you see 2 - 3 - n packages with zend/* you are now sure that "zend" it's their vendor name, because you seen that on zend website.

My question is how you tell a tool that can trust example.com for anything related to zend, because it's their website ?

Like I said before:

[John Patrick Gerdeman]

The current approach is NOT to create a central repository for security advisories. The standardization COULD result in third parties aggregating them.

The approach currently is:
1. Define a file to be included in the project's repository.
2. This file either contains the information itself or a url where it can be fetched from.
3. Define a standard representation

>> How do you make sure that a submitted URL really belongs to that project ?

1. The information comes from the project's own repository.
You'd have to create a fake repository and point to a wrong address for no gain whatsoever. That already is a strong deterrent. [1]
2. Any kind of moderation measures are outside of the FIG's scope.

[1] In theory I could fork a project and change the address. Users that use the fork could then be fed false information introducing exploitable weaknesses. But then I could've just changed the code itself. I don't believe this is a relevant scenario.

~John

[Alexandru Guzinschi]

On Sunday, November 2, 2014 10:16:04 AM UTC+2, John Patrick Gerdeman wrote:

>> How do you make sure that a submitted URL really belongs to that project ?

1. The information comes from the project's own repository.
You'd have to create a fake repository and point to a wrong address for no gain whatsoever. That already is a strong deterrent. [1]
2. Any kind of moderation measures are outside of the FIG's scope.

My question is related to Pádraic Brady's statement:

 

On Friday, October 31, 2014 3:43:00 PM UTC+2, Pádraic Brady wrote:

 

Any sort of service as you envision only needs one thing - a set of project URLs. It can pull those from Packagist (if you can backtrack how its API works - not hard), crawl git repositories, accept anonymous URL submissions, etc. 

More precise to "accept anonymous URL submissions".

[Pádraic Brady]

On 2 November 2014 09:14, Alexandru Guzinschi <al...@gentle.ro> wrote:

On Friday, October 31, 2014 3:43:00 PM UTC+2, Pádraic Brady wrote:

 

Any sort of service as you envision only needs one thing - a set of project URLs. It can pull those from Packagist (if you can backtrack how its API works - not hard), crawl git repositories, accept anonymous URL submissions, etc. 

More precise to "accept anonymous URL submissions".

Let's say there is a central repository service for vulnerability reporting. I go there, and submit the Zend Framework URL. That's all I'm submitting. Obviously, in that case, the service will need to perform some level of verification. For example, fetch the canonical git URI & composer.json, cross-reference this to Packagist, verify that the identity assertion is in agreement between the retrieved composer.json and Packagist data, and only then accept vulnerability data from that source. This is all easily automated.

It would, of course, fail completely if Packagist data were to become corrupted or untrustworthy. So such a service would need to do periodical audits of its data checking for discrepancies between that data and Packagist. This would detect any Packagist corruption, identify deleted projects to stop crawling, highlight revised canonical data (e.g. were a project to move or rename itself), etc. And you need someone to action audit results where automation might be unwise...

Again, implementation is beyond the scope of this group so this is merely what I'd consider as a starting point were I to be building something like this.

[Alexandru Guzinschi]

On Sunday, November 2, 2014 12:07:44 PM UTC+2, Pádraic Brady wrote:

It would, of course, fail completely if Packagist data were to become corrupted or untrustworthy. 

 
Or if packagist will become unavailable.

Thanks for taking the time to clarify that aspect Brady. Now makes more sense to me.

[Lukas Kahwe Smith]

Aloha,

I would like to try out a new approach to ironing out a proposal for this topic.
I would like to create a dedicated mailinglist for this where half a dozen max people that are interested have rights to post. The list itself would be public however. Once we have a first draft we would come back to this list to discuss it (or maybe we will just temporarily give public post rights to the new list). I am hoping that this will allow us to more efficiently move forward.

Obviously any people can create a mailinglist to discuss things as in the end if it becomes a PSR it will have to go through the normal voting process anyway, but I just want to make sure that people are aware of this “experiment” and nobody feels left out. But at the same time I do not think a formal vote is necessary to conduct such an experiment. Furthermore I am obviously soliciting people to raise their hand who want to be added to this mailinglist.

Now the big question is how to best get to the proposal stage to get a PSR number assigned. it seems sensible to name the mailinglist php-fig-psrX@. also I guess the proposal stage (https://github.com/php-fig/fig-standards/blob/master/bylaws/004-psr-workflow.md#21-pre-draft) is sort of a good way to ensure that we are at least on to something before we open a mailinglist. so I guess this initial step should happen on this list after all. for now I have created a PR with an initial proposal:
https://github.com/php-fig/fig-standards/pull/368

regards,
Lukas Kahwe Smith
sm...@pooteeweet.org

[Pádraic Brady]

Hi Lukas,

On 5 November 2014 09:59, Lukas Kahwe Smith <sm...@pooteeweet.org> wrote:
> Aloha,
>
> I would like to try out a new approach to ironing out a proposal for this topic.
> I would like to create a dedicated mailinglist for this where half a dozen max people that are interested have rights to post. The list itself would be public however. Once we have a first draft we would come back to this list to discuss it (or maybe we will just temporarily give public post rights to the new list). I am hoping that this will allow us to more efficiently move forward.

And reduce "spam" ;).

> Obviously any people can create a mailinglist to discuss things as in the end if it becomes a PSR it will have to go through the normal voting process anyway, but I just want to make sure that people are aware of this “experiment” and nobody feels left out. But at the same time I do not think a formal vote is necessary to conduct such an experiment. Furthermore I am obviously soliciting people to raise their hand who want to be added to this mailinglist.

I'd love to be a member and help contribute.

> Now the big question is how to best get to the proposal stage to get a PSR number assigned. it seems sensible to name the mailinglist php-fig-psrX@. also I guess the proposal stage (https://github.com/php-fig/fig-standards/blob/master/bylaws/004-psr-workflow.md#21-pre-draft) is sort of a good way to ensure that we are at least on to something before we open a mailinglist. so I guess this initial step should happen on this list after all. for now I have created a PR with an initial proposal:
> https://github.com/php-fig/fig-standards/pull/368

Would be great if two voting representatives would agree to sponsor in
that case, with one acting as cooridinator (i.e. glorified vote
counter ;)). I'm already sponsoring and coordinating two PSRs, and I'm
sure Larry would agree that getting more members involved in PSRs is
very much desireable.

Paddy

[Larry Garfield]

:-)

I'd be interested in participating, and would be willing to Sponsor
(something I'm not doing elsewhere).

--Larry Garfield

[Korvin Szanto]

I’d love to sponsor this as well.

--
Korvin Szanto

--
You received this message because you are subscribed to the Google Groups "PHP Framework Interoperability Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+u...@googlegroups.com.
To post to this group, send email to php...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/546055B6.5080806%40garfieldtech.com.
For more options, visit https://groups.google.com/d/optout.

[Pádraic Brady]

I count two sponsors ;). Would one of you be willing to coordinate
votes and any eventual Review stage?

Paddy

> https://groups.google.com/d/msgid/php-fig/etPan.546135f2.625558ec.2f8%40chuck-testa-3.local.

>
> For more options, visit https://groups.google.com/d/optout.

--

[Korvin Szanto]

I’ve decided to coordinate. I’ve opened entrance voting.

--
Korvin Szanto

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/CALwr1G%3D-b_%3DLZ46f-G93sEAq0bG_wa4bQeyK5J2Vs0pjF_%2BaZw%40mail.gmail.com.

[Michael Cullum]

I'd be interested in participating in the break-away discussions about this also.

--
Thanks,

Michael C
phpBB

[dar...@enshrined.co.uk]

Hi Guys,

This sounds like a great idea and something I'd love to get involved in.

I fully agree with staying away from implementing anything and just specifying a standard for releases. I think it makes much more sense to leave implementation details up to the various applications that want to use the data and to stick with just standardising the format that it'll be released in.

Cheers,

Daryll Doyle

[Lukas Kahwe Smith]

> On 10 Nov 2014, at 22:20, Pádraic Brady <padrai...@gmail.com> wrote:
>
> I count two sponsors ;). Would one of you be willing to coordinate
> votes and any eventual Review stage?
>
> Paddy
>
>> On 10 November 2014 22:02, Korvin Szanto <korvin...@gmail.com> wrote:
>> I’d love to sponsor this as well.

>> On November 9, 2014 at 10:05:48 PM, Larry Garfield (la...@garfieldtech.com)
>> wrote:
>>
>> I'd be interested in participating, and would be willing to Sponsor
>> (something I'm not doing elsewhere).
>

Awesome!

I am on vacation until the 17th.

BTW: I guess we will also need to figure out if this all should end up being two PSRs or just one. but I guess we do not need this cleared up just now. Once we have the official proposal stage we at least get one PSR number assigned from my understanding and if necessary we can split out another PSR proposal later on.

regards,
Lukas

[Dracony]

It starts to look a lot more like a Packagist feature than a PSR. All that is needed to implement this is a specific parameter in composer.json

 

vulnerabilities:{
  '<=1.0.12': ['http://somesite.com/new-vulnerability-found', ...]
}

Then composer would give a warning if a vulnerable version is installed and Packagist would have a page with latest vulnerabilities found.

How come does this need a PSR ? It belongs in a feature request in Composer/Packagist

On Thursday, October 30, 2014 6:48:42 AM UTC+1, Lukas Kahwe Smith wrote:

Aloha,

[Timo H]

@Dracony, take a look at Pádraic's blog post which sums up the needs behind this PSR really well: http://blog.astrumfutura.com/2014/11/security-oriented-psr-proposed-to-php-fig/

Timo

[Dracony]

Just read it the post and it really didn't convince me that an abstract security policy ( akin to a style guide) would function better than a Packagist/Composer feature. Perhaps an example of such an existing policy or a t least a mini draft would be more convincing

[William Durand]

> Just read it the post and it really didn't convince me that an abstract security policy ( akin to a style guide) would function better than a Packagist/Composer feature. Perhaps an example of such an existing policy or a t least a mini draft would be more convincing

A Composer/Packagist feature is an implementation detail to me. We first need a standardized security policy, and based on that, we could think about augmenting existing tools.

--

William

--

You received this message because you are subscribed to the Google Groups "PHP Framework Interoperability Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+u...@googlegroups.com.
To post to this group, send email to php...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/8963f1ba-6572-4899-8341-503392b47f6e%40googlegroups.com.

[Pádraic Brady]

On 12 November 2014 13:47, Dracony <draco...@gmail.com> wrote:
> Just read it the post and it really didn't convince me that an abstract
> security policy ( akin to a style guide) would function better than a
> Packagist/Composer feature. Perhaps an example of such an existing policy or
> a t least a mini draft would be more convincing

I think you may be missing the point, so to clarify:

1. A security policy is a text document published on a website which
outlines how an organisation intends handling security
vulnerabilities. That's a simple definition - you can document a lot
more in a policy. It would assist in setting common standards,
expectations and eliminating doubt between the three typical parties
to a vulnerability report - programmer, reporter and user.
2. In order to share vulnerability disclosures more effectively, we
need to have an agreement on the minimum data required, optional data
if needed, and the format of that data (i.e. JSON/YAML/XML). Issues
like location, and the discovery of the location, may also fall under
that agreement. This ignores HOW the data eventually gets used since
that's a problem for third parties writing tools, continuous
integration platforms, and other such possible consumers, though
they're more than welcome to check in on the discussion and hatch
plans.

If you're looking for examples of policies, Google is stuffed with
them. There's even a pre-existing vulnerability database
(centralised): https://github.com/FriendsOfPHP/security-advisories and
tooling that uses it in real life. This PSR is not being born in a
complete vacuum.

Paddy

[Roman Tsjupa]

Well, if the vulnerabily thing gets implemented into the composer as I suggested the FriendsOfPHP security advisory you linked to will become absolete. Since it does exactly that: checks your composer.json against a local database. If we store vulnerability info in the packages composer.json files there will be no need for such a database at all.

Instead of JSON/YAML/XML stuff all that's really needed is a link to a detailed description page, which I also suggested as part of the composer implementation. And even if we absolutely must have some data lilke 'time' it still can be embedded into composer.json.

Vulnerability info is metadata. We already store a lot of metadata like license and stuff in the composer file. Why come up with a new bicycle, just put it there like everything else ?

--
You received this message because you are subscribed to a topic in the Google Groups "PHP Framework Interoperability Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/php-fig/45AIj5bPHJ4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to php-fig+u...@googlegroups.com.

To post to this group, send email to php...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/CALwr1GmD98c-S6YhW_CSnfH1Znb6B2ooUYbwhsfEYy25L2svow%40mail.gmail.com.

[Daryll Doyle]

The way I see it, is that it shouldn't be tied to composer. It should be a standalone file/s that if composer want to pull in, they can. As can any other third party implementation.

What if a package wants to publish vulnerability data but they're not using composer? They are then once again left out in the cold and will have their own format and data standards, therefore ending up pretty much back at square one.

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/CANamvv1J5N182BL6feNirtQSckv%3D19V1hDrCDt9C56V6G5GBkA%40mail.gmail.com.

[Dracony]

In that case I would suggest keeping the JSON format for that file too. Let's at least be consistent with whats already out there =)

[Larry Garfield]

It's also the machine-readable format OF the content at that URI, which allows services to actually read and parse the data.  That the URI may be distributed by Composer files (or other mechanisms) is not the task at hand.

--Larry Garfield

--

You received this message because you are subscribed to the Google Groups "PHP Framework Interoperability Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+u...@googlegroups.com.

To post to this group, send email to php...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/8963f1ba-6572-4899-8341-503392b47f6e%40googlegroups.com.

[Dracony]

Is there some work going on on this? I haven't noticed any updates recently? Is there a separate mailing list?

[Lukas Kahwe Smith]

> On 10 Dec 2014, at 10:23, Dracony <draco...@gmail.com> wrote:
>
> Is there some work going on on this? I haven't noticed any updates recently? Is there a separate mailing list?

Yes there is a separate list:
https://groups.google.com/forum/#!forum/php-fig-psr-9-discussion

Note its invite only for posting but everyone can read.