Verified Software Competition - second announcement - http://vscomp.org
38 views
Skip to first unread message
Hannes Mehnert
Apr 18, 2013, 4:02:50 AM4/18/13
to idris...@googlegroups.com
(first announcement available at
http://lists.seas.upenn.edu/pipermail/types-announce/2013/003699.html)
Second Announcement
VSTTE Competition 2013
20-22 April 2013
Organizers: Joseph Kiniry, Hannes Mehnert, Dan Zimmerman
This edition of the VSTTE programming contest is an experiment of a
different kind, as it is more about software engineering than
programming. It is not a contest to see who can write and verify small
problems as quickly as possible, but instead how can a team create a
quality piece of code, using any tools and techniques (not just
verification), in a short period of time.
Quality software is about more than just verified data types and
algorithms at the source code level. Unlike previous competitions, this
year�s VSComp will focus on a rigorously engineered software system.
Contestants will be evaluated for all of the software engineering
artifacts that they produce, not just for verifying their implementations.
Consequently, teams that competed in previous competitions are
encouraged to recruit new team members whose skills complement those of
the existing team members. For example, perhaps the current team is
great at low-level design and verification, but is weak in writing
requirements or in rigorous validation/testing.
The aims of the competition are:
to bring together those interested in rigorous software engineering
and formal verification, and to provide an engaging, hands-on, and fun
opportunity for competition and mutual-learning,
to evaluate the usability of a variety of software engineering
tools, not the least of which are logic-based program verification
tools, in a controlled experiment that could be easily repeated by others.
After the initial announcement of the contest we were asked the
following question by more than one party. You�ll find it in our FAQ.
Q. Why attach this style of contest to VSTTE, if we are not focusing
exclusively on verification?
A. Not a single international programming contest rewards good software
engineering behavior. We have tried for years to influence the big
contests and venues (e.g., ACM, ICFP, and TopCoder) to pay more
attention to engineering and quality, but to little effect. If we are
going to see a contest like this survive and even thrive in the long
run, it seems it has to be an outgrowth of the verification community,
rather than the implementation-centric hack-fast community.
The contest takes place over a two-day period. The system that
contestants must develop is secret until the moment the contest starts.
The system will be decomposed for the contestants into an architecture,
whose constituent pieces are the sub-problems of the contest. Thus, by
solving all sub-problems, one writes the entire application. What�s
more, the architecture is specified in such a way that independent
solutions to sub-problems submitted by competing teams should compose
into the final system.
The kinds of software engineering concepts mentioned in the contest
include: requirements, domain analysis, design, architecture, formal
specifications, implementation, validation, verification, and
traceability. A well-prepared team will have a methodology prepared for
each of these facets. The submission of a solution for a sub-problem
need not include any of these facets in particular�i.e., running,
verified code is neither necessary nor sufficient to win the contest.
There are no restrictions on concepts, tools, and technologies used.
Teams whose focus in on �early� (i.e., requirements or domain analysis)
or �late� (validation/testing or evolution) phases of the software
engineering process are very welcome. There is no limit on team size,
but the results will be normalized by team size.
We particularly encourage participation of:
student teams (this includes PhD students),
non-developer teams using a tool someone else developed, and
several teams using the same tool
A panel of judges will evaluate contest entries to independently score
sub-problems and determine the winner. Solutions will be judged for
correctness, completeness and elegance. The total score for a
sub-problem is the sum of its scores in the following categories, where
the total number of points in each category available is indicated in
parentheses: domain analysis (3), requirements (3), architecture (3),
design (6), implementation (6), validation (6), formal verification
(12), traceability (3). The maximum number of points available for each
sub-problem is 42. The verification researcher will note the weight
given to formal verification.
All submitted artifacts will be made public immediately after the
contest ends so that contestants can comment upon each other�s
submissions. We expect that a paper will be co-authored by all
interested contestants about the contest�s results, as in several
previous contests.
The contest begins at 9:00 GMT on Sat 20 April and ends at 9:00 GMT on
Mon 22 April.
Prizes will be awarded in the following categories:
best team
best student team
tool used most effectively by the most teams
The contests website is http://www.vscomp.org/. You will find there
momentarily an outline of the contest, a some Frequently Asked
Questions. The contest problems will go live at this site at 9:00 GMT
on Saturday the 20th of April. (Take note of your daylight savings time
GMT offset!)
Questions or comments about the contest should be sent to Joe Kiniry
(kin...@acm.org).
http://lists.seas.upenn.edu/pipermail/types-announce/2013/003699.html)
Second Announcement
VSTTE Competition 2013
20-22 April 2013
Organizers: Joseph Kiniry, Hannes Mehnert, Dan Zimmerman
This edition of the VSTTE programming contest is an experiment of a
different kind, as it is more about software engineering than
programming. It is not a contest to see who can write and verify small
problems as quickly as possible, but instead how can a team create a
quality piece of code, using any tools and techniques (not just
verification), in a short period of time.
Quality software is about more than just verified data types and
algorithms at the source code level. Unlike previous competitions, this
year�s VSComp will focus on a rigorously engineered software system.
Contestants will be evaluated for all of the software engineering
artifacts that they produce, not just for verifying their implementations.
Consequently, teams that competed in previous competitions are
encouraged to recruit new team members whose skills complement those of
the existing team members. For example, perhaps the current team is
great at low-level design and verification, but is weak in writing
requirements or in rigorous validation/testing.
The aims of the competition are:
to bring together those interested in rigorous software engineering
and formal verification, and to provide an engaging, hands-on, and fun
opportunity for competition and mutual-learning,
to evaluate the usability of a variety of software engineering
tools, not the least of which are logic-based program verification
tools, in a controlled experiment that could be easily repeated by others.
After the initial announcement of the contest we were asked the
following question by more than one party. You�ll find it in our FAQ.
Q. Why attach this style of contest to VSTTE, if we are not focusing
exclusively on verification?
A. Not a single international programming contest rewards good software
engineering behavior. We have tried for years to influence the big
contests and venues (e.g., ACM, ICFP, and TopCoder) to pay more
attention to engineering and quality, but to little effect. If we are
going to see a contest like this survive and even thrive in the long
run, it seems it has to be an outgrowth of the verification community,
rather than the implementation-centric hack-fast community.
The contest takes place over a two-day period. The system that
contestants must develop is secret until the moment the contest starts.
The system will be decomposed for the contestants into an architecture,
whose constituent pieces are the sub-problems of the contest. Thus, by
solving all sub-problems, one writes the entire application. What�s
more, the architecture is specified in such a way that independent
solutions to sub-problems submitted by competing teams should compose
into the final system.
The kinds of software engineering concepts mentioned in the contest
include: requirements, domain analysis, design, architecture, formal
specifications, implementation, validation, verification, and
traceability. A well-prepared team will have a methodology prepared for
each of these facets. The submission of a solution for a sub-problem
need not include any of these facets in particular�i.e., running,
verified code is neither necessary nor sufficient to win the contest.
There are no restrictions on concepts, tools, and technologies used.
Teams whose focus in on �early� (i.e., requirements or domain analysis)
or �late� (validation/testing or evolution) phases of the software
engineering process are very welcome. There is no limit on team size,
but the results will be normalized by team size.
We particularly encourage participation of:
student teams (this includes PhD students),
non-developer teams using a tool someone else developed, and
several teams using the same tool
A panel of judges will evaluate contest entries to independently score
sub-problems and determine the winner. Solutions will be judged for
correctness, completeness and elegance. The total score for a
sub-problem is the sum of its scores in the following categories, where
the total number of points in each category available is indicated in
parentheses: domain analysis (3), requirements (3), architecture (3),
design (6), implementation (6), validation (6), formal verification
(12), traceability (3). The maximum number of points available for each
sub-problem is 42. The verification researcher will note the weight
given to formal verification.
All submitted artifacts will be made public immediately after the
contest ends so that contestants can comment upon each other�s
submissions. We expect that a paper will be co-authored by all
interested contestants about the contest�s results, as in several
previous contests.
The contest begins at 9:00 GMT on Sat 20 April and ends at 9:00 GMT on
Mon 22 April.
Prizes will be awarded in the following categories:
best team
best student team
tool used most effectively by the most teams
The contests website is http://www.vscomp.org/. You will find there
momentarily an outline of the contest, a some Frequently Asked
Questions. The contest problems will go live at this site at 9:00 GMT
on Saturday the 20th of April. (Take note of your daylight savings time
GMT offset!)
Questions or comments about the contest should be sent to Joe Kiniry
(kin...@acm.org).
Reply all
Reply to author
Forward
0 new messages