Oops, my mistake. Three quotes is right. This works for me:
drop table if exists foobar;
create table foobar (name varchar(20));
insert into foobar (name) values ('''steve''');
Furthermore, if this is being generated by Java code, then you shouldn't be doing it with string concatenation.
You should be doing this:
Not only does this let the JDBC driver take care of the work of escaping the single quotes at the beginning and end of the string, it is also safer against SQL Injection attacks.
// assuming url contains the correct url for your H2 database...
final Connection connection = DriverManager.getConnection(url);
PreparedStatement statement = connection.prepareStatement("insert into foobar (name) values (?)");
statement.setString(1, "'Steve'");
statement.executeUpdate();