A test fork of Firebird

[Dalton Calford]

Hey all,

I have been working on a fork of an earlier snapshot of the version 6 codebase.

I have renamed it specifically so that there is no confusion between firebird and the fork.

I am using a combination of multiple AI agents and testing processes to add perform tests and add features.

I have been adding features, running tests, having it compile (in linux at least) and basically playing.

I have not made a release but I am putting the code up in git as I progress through the different stages.

https://github.com/DaltonCalford/ScratchBird

I will likely work on the python driver for it before I make a release, otherwise it is just sb_isql (yes I even renamed the tools so that there will be no name conflicts)

I git-commit/sync every time I get up from my desk, so if you download, you may get a version that compiles or not, once I do a release, the release version will be able to compile.

I am just putting this here to let you all know it exists. Some of the stuff may seem interesting to you.

regards

Dalton

[Mark Rotteveel]

On 14/07/2025 18:11, Dalton Calford wrote:
> Hey all,
>
> I have been working on a fork of an earlier snapshot of the version 6
> codebase.
>
> I have renamed it specifically so that there is no confusion between
> firebird and the fork.
>
> I am using a combination of multiple AI agents and testing processes to
> add perform tests and add features.
>
> I have been adding features, running tests, having it compile (in linux
> at least) and basically playing.
>
> I have not made a release but I am putting the code up in git as I
> progress through the different stages.
>

> https://github.com/DaltonCalford/ScratchBird <https://github.com/
> DaltonCalford/ScratchBird>

>
> I will likely work on the python driver for it before I make a release,
> otherwise it is just sb_isql (yes I even renamed the tools so that there
> will be no name conflicts)
>
> I git-commit/sync every time I get up from my desk, so if you download,
> you may get a version that compiles or not, once I do a release, the
> release version will be able to compile.
>
> I am just putting this here to let you all know it exists. Some of the
> stuff may seem interesting to you.

Out of curiosity, what is the difference of "PascalCase Object
Identifiers" compared to regular (unquoted) identifiers? Based on the
example in the README, it behaves the same.

Mark
--
Mark Rotteveel

[Dalton Calford]

Hey Mark, 

With quoted identifiers, you must always use the quotes and use the same case .

With Pascal case, the object name is displayed as mixed case, but does not care what mixture of case you use to refer to it.

With quoted "mytable"<>"MyTable" but with Pascal case, mytable=MyTable.

It's a small thing but an appreciated thing.  It's 

One of the little things that mssql does that is actually not a pain in his backside.

Regards 

Dalton 

--
Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate
---
You received this message because you are subscribed to the Google Groups "firebird-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-deve...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/0512702f-8ffb-43c1-9c7f-2127851a9f06%40lawinegevaar.nl.

[Mark Rotteveel]

On 15/07/2025 10:48, Dalton Calford wrote:
> Hey Mark,
>
> With quoted identifiers, you must always use the quotes and use the same
> case .
> With Pascal case, the object name is displayed as mixed case, but does
> not care what mixture of case you use to refer to it.
> With quoted "mytable"<>"MyTable" but with Pascal case, mytable=MyTable.
> It's a small thing but an appreciated thing.  It's
> One of the little things that mssql does that is actually not a pain in
> his backside.

So it's more an always case-insensitive identifier (and store as is)
thing than a "PascalCase" thing.

In any case, I'd suggest altering your example to use quoted identifiers
to illustrate the use case better (because regular identifiers are
case-insensitive in Firebird, they're just stored uppercase.

Mark
--
Mark Rotteveel

[Dalton Calford]

Most of that was generated by the AI.   I did not really care so I paid more attention to try to get it to stop saying its ready for use.   

I don't care what AI you are using, they all have that all annoying "Everything is Wonderful, You are so Smart" response even if you are about to bring your machine to its knees.

Right now, the stuff I am paying attention to is the documentation and code - AI can generate so much stuff that it is a full time job keeping up with it.   
I start questioning how to do a particular task in the code base, what it would impact, what metrics would show the impact of the change the most given the way it interacts with other elements etc., you know, the basic getting the feel type questions to decide IF you want to do something, but, before I know it, it has a full specification and implementation document generated in the root of the project.  (btw, the previous sentence proves why I do need ai to stop my run on sentences)

I figured the only people really looking are those who want to see what is implemented, and how it was implemented.   So the source was the key to get out the door first.  The Readme is more of a guide for the code readers to understand what I have done, thus knowing where to look to find the code changes.

For those who are just curious, then the colorful self-promoting text does not really matter.  I see it more as a way to have people ask questions like you did, or ask about features I never thought of.

As this is a test bed I can try things for no other reason then to see what would happen, which you can not really do with a production codebase, as you would need to create a separate build environment, then keep track of where you did what.  Its possible, but more of a hassle than it's worth.

best regards

Dalton

--
Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate
---
You received this message because you are subscribed to the Google Groups "firebird-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-deve...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/90d86cee-839e-4f34-ae0a-899c97881e1d%40lawinegevaar.nl.

[Dalton Calford]

For those who are curious about the actual state of the build, vs the a** kissing AI generated README.md, check out

• COMPILATION_METHODS.md

• RELEASE_BLOCKERS.md

• TECHNICAL_DEBT.md

These are in the root of the project and detail the various elements that I am currently addressing.

There are others, but these are the must do's before I release my test stack of sb_isql sql files.

Best regards

Dalton

[liviuslivius]

Hi

Just out of curiosity, why aren't you doing this in the main branch as a series of pull requests, but instead as a separate repo?

Let me know if you'd like it phrased more formally or casually!

Regards,

Karol Bieniaszewski

-------- Oryginalna wiadomość --------

Od: Dalton Calford <dalton....@gmail.com>

Data: 15.07.2025 13:56 (GMT+01:00)

Do: firebir...@googlegroups.com

Temat: Re: [firebird-devel] A test fork of Firebird

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/CANY9GwrhiFiQGkRwLTpVO9pVhstPBTj7Oe8R-QkP%3D%2BVxGpuo%3Dw%40mail.gmail.com.

[liviuslivius]

Sorry for copy paste too much on the phone ;-)

Regards,

Karol Bieniaszewski

-------- Oryginalna wiadomość --------

Od: liviuslivius <livius...@poczta.onet.pl>

Data: 15.07.2025 17:51 (GMT+01:00)

[Dalton Calford]

No worries.

First, I am using AI as a coding assistant - it does randomly stupid things (I am using a few but claude and gemini-cli are examples) such as "fixing" problems by commenting out code and putting stubs in its place.  It can also remove or summarize comments, Automatically change code (and thus code behaviour) without notice etc., This is not something to put into the main codebase, I wouldn't even try.  Once a workable system shows how to do it, the same idea can be ported over, long after testing shows it is a good idea to even try.

Second, I am adding features that are not SQL standard, such as recursive schema's and data types/index types that I use but are not in the core firebird product.   Combine this with what could be a very long development cycle and by the time I am ready for a push/pull request, the codebases would be so divergent, that I might as well have done what I actually did do, fork the project so that I don't have to worry about breaking things for other people or having a constantly changing code base that I am trying to modify.

Third,  this is intended as a way for me to play - a hobby.  I don't want the pressure of having to support the various extensions I am adding.

Now, since the stuff I am adding is based upon the firebird codebase, you guys can look at what I have done, decide if it is a good idea or not, and since the source is available, implement it yourselves. 

It was a decision I did not take lightly, as I love Firebird and want everyone using it, but, for the sake of this project, it's better for the projects to be separate.

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/4bhNyG6wrpz1y98%40smtp.poczta.onet.pl.

[Piergiorgio Valli]

Hi All,

I'm amazed by the amount of code suggested and modified via AI, but it's also thanks to Dalton's extensive experience.

It's an interesting way to experiment with new features.

Best regards
Piergiorgio Valli

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/CANY9GwoDsT%2ByuJdYbsQ0v9wqhHMmDyPUXiyQhO0PjHkboKbqNQ%40mail.gmail.com.

[Dalton Calford]

And just to say, I am open to anyone giving ideas for what to add (I already am working on a second project that is a vector database index store as well as links into llm's for tag generation)

The ultimate goal is a database that is based upon Dynamic N6 datastructures that (by its design) can support graph databases.   I have already ported open-cython (a language for working with graph databases) as a UDR written in pascal before starting on this journey.   The one thing that I have going for me is that right now, I have alot of free time, I have experience with the four core types of nosql databases and how they are designed at a fundamental level, and thanks to my ADD/OCD (hyperfocus anyone??) and other mental ailments (yes I am nuts, its what has gotten me this far) I have a little more experience than others in regards to databases.

My first stages was to get ScratchBird up to competition with the other major contenders.  Which meant changing page sizes to 128k so that I can extend the row size to a more reasonable level.   I also started adding data types and features that I personally wanted (such as unsigned integers and Pascal/Camel Case support)   I also specifically extended the schemas to be act more like a directory tree - or namespace if you prefer.   So, yea, its fun, its currently working (I had the hardest time trying to get gpre to compile without existing databases and the code ignored the manual flag - I got around that by building the server first as part of the bootstrap, then have the server generate the needed databases that gpre wants).   I am still working on a few issues before I make a .5 official release.  I don't really suggest anyone use that, as I am doing a total redesign of the schema layout in the .6 codebase.

Regardless, 

I hope you guys have as much fun watching and kibitzing about the stuff I am doing as I am doing it.

best regards

Dalton

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/CALJ6nywNFGGPLk%2BFg6OFk9OxdM0gH-NtULkOyo6LgEoGOxo5ew%40mail.gmail.com.

[Dalton Calford]

Just read my response and realized I mistyped - openCypher not open-cython.   Too much on the brain lately.

[Pavel Cisar]

Hi Dalton,

I think that it's truly awesome what you are doing. It doesn't actually
matter if any code you create on your discovery journey will make it
into Firebird anytime soon, as it's the first hand experience you are
getting that matter most.

I think that in future (and not very distant it seems), we could use AI
coding assistants for some precisely targeted tasks on Firebird code, as
separate, focused PR. Generating documentation, comments, localized
refactoring or test creation seems like good first candidates. And for
that, your pioneering experience would be invaluable. It would be great
if you decided to blog and record your journey. Your observations,
discoveries what works or don't, prompt engineering strategies, context
management, model differences etc. It would certainly be an interesting
read for anyone interested in coding with the help of AI agents, and a
great help for those who would like to try it out for themselves
specifically with Firebird or related projects. BTW, are you using
Cursor, or VC Code with plugins like Cline? Or something else?

For example, I already started to experiment with gemini 2.5 on Firebird
Python projects (driver etc.) with some interesting results. It could be
interesting to compare our findings.

best regards
Pavel Cisar
IBPhoenix

[Dalton Calford]

Hi Pavel,

Thanks for the words of encouragement - I was worried that my little sanity project (ie to keep me sane) would be taken the wrong way by the community.

Since March, I have tested quite a few different models.  I am a linux guy and had a series of accumulated laptops that had APU's (combined CPU and GPU) and APU's are the magic for setting up your own AI lab.

You see, AI needs processing power and GPU's are effectively a bank of parallel custom cpu's.  BUT the GPU needs to be able to address the memory - thus the need of having a series of gpu's with large amounts of ram - this can get VERY expensive in short order.

Well, APU's are designed to be small, power efficient and don't have their own ram -they share the ram with the CPU part of the chip.   So if you have 64GB of ram in your laptop, you can dedicate about 48GB to your video card (more if you use linux, windows is a memory hog).   With ollama, you can load a llm model into video memory and use the GPU/APU for your processing.   IF you have a decent lan, well you can even split the model across machines.   So I networked a bunch of old gaming laptops, to a 10GB lan (most of the laptops could only handle 1GB but that was fine)   And I used a "Quantised"  version of a few different open source LLMs.    It was slow,(really slow) but it worked.  

These days, you can buy motherboards with decent APU's and 128GB of DDR5 ram for the price of a mid range video card (https://frame.work/ca/en/products/framework-desktop-mainboard-amd-ryzen-ai-max-300-series?v=FRAFMK0006)

The latest models (almost a TB in size) could be spread across a series of 9" racks (about 10-12 mainboards) for the cost of a single server pizzabox.  This would be fast and (relatively) cheap

Anyways, back to the topic at hand.

I started with Gemini in the web interface, I would ask for a single pascal file that had one or two specific functions or objects in it - I would combine that in FPK (lazarus) and build my project in bits and pieces.  This was good to a point, but, it was slow, tedious and with a large project, the AI would forget what it did before and start to make crap up.

So, I tried VS code, going through various agents for different ai's, and I was never happy with the result.

So, I tried claude (command line ai that can access all the files in the directory it is started in and any of the sub-directories) and gemini-cli - similar to claude but not as good in some things and better in other areas.

So, now start by telling the AI to analyse the code base and put the results of their analysis into a markdown file (*.md).   Now, when you want to refresh the ai, instead of it burning through tokens and context trying to understand what you are working on, you tell it to load the details from the markdown file.   There are many ways to do this, but, you get the idea.

Now, you have to ask a series of questions, have the ai banter with you over the best ways to approach a problem, BUT BE SURE to tell them to report and not act upon the discussion - many times I am discussing a plan of action - building a living specification, using the AI to research the code base or alternative methods of doing something, when all of a sudden, it just goes and starts changing your files, going down it's own path and before you know it, you lose all your work, all your context and wasted your tokens.  This is why I commit everything, even broken code and sync it with github - with a single random occurrence, I could lose my whole codebase.

So, to answer your question, I use the command line and interactive prompts.   I let the AI go in and play with the entire code base so if it has a question, it can look it up itself instead of guessing.   I have to monitor everything it is doing - hours of staring at scrolling text, seeing the output of compiles, random error messages to understand what the AI is doing, trying to figure out if it is going rogue or not.

It is long hard hours of work, trying to stop the AI from wrecking things, BUT, the result is better than MONTHS of work and you get to completely guide the whole process.

A non-programmer could get a program made, but the program will be brittle, have holes and do indeterminate actions.  

You need a programmer to understand what is happening, produce a proper specification, know what questions to ask and how to understand the results.

AI is a game changer, but, it is just a tool like any other.

--
Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate
---
You received this message because you are subscribed to the Google Groups "firebird-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-deve...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/78bfc9ab-550f-4f7a-b8be-71b749c82bc9%40gmail.com.

[Jim Starkey]

Aide from the well know pitfalls of AIS generated code (as it, doesn't work), keep in mind that any code that it generates was probably cobbled from something it picked up on the web, and there is a very good chance that it was cobbled from a GPLed project.  If that is the case, you run risk of compromising the Firebird license with GPL.

In other words, you're dumpster diving without the original headers, copyrights, licenses, and restrictions.

Companies that use that crap internally may well choose to the risk.  On the other hand, any organization that distributes code would be profoundly mistaken to use anonymous code taken from an unknowable site. 

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/CANY9GwoDsT%2ByuJdYbsQ0v9wqhHMmDyPUXiyQhO0PjHkboKbqNQ%40mail.gmail.com.

--
Jim Starkey

[Dalton Calford]

A few months ago, I would have agreed with you.   But the models, especially the new reasoning models ,  could be a TB in size and still not hold everything they are accused of plagerizing.  

A year ago, with chat gpt4 I would have agreed with you. It could only answer 2+2 because it had been trained on the pattern to answer 4.

The current models, are smaller than chat gpt4, but far better.  The technology is changing exponentially.  It's no longer auto complete, but it doesn't think either.

For example, most ai's know of firebird, but they get the dialect wrong, it's close but, when it comes to generated procedures, or even simple tables, it is broken.  But, load the firebird pdf into the environment, and tell it to analyze it, you get proper working dml/ddl/pl code and not just simple code, I am talking a recursive n6 many to many design.

The thing that sold it to me was when a buddy of mine at work, created a whole programming language, just as a pdf.   He wanted to prove that ai is just auto complete.   So with a language that was partially written in Klingon, using reversed, reverse polish notation (read it right to left) and some structures from forth, COBOL and basic, he presented the text specification to the ai along with some fpk code and told the ai to convert the fpk code into his made up language.   So we are talking about no example code other than a specification and it did it.

If it wasn't for the prize of a case of beer (Canadian, none of that water they call beer south of the border). We would have never found out, but three of the guys took a week after work and on the weekend to actually read and debug the mess that was generated.  It was correct.

That was the Gemini 2.5 pro web interface that did that.

Gemini is not even the best.  AI is something, not auto complete, not intelligent, but something.

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/0b6b714e-86e6-490d-aeff-ce4f4a108eb1%40jimstarkey.net.

[Dalton Calford]

May I ask why we use GPRE?  it seems kinda brittle and finicky - Is there a purpose that I am not seeing?

[Dimitry Sibiryakov]

Dalton Calford wrote 16.07.2025 16:17:
> May I ask why we use GPRE?  it seems kinda brittle and finicky - Is there a
> purpose that I am not seeing?

It takes as the input a code that is simpler than direct calls to VIO
primitives and produces a code that is faster than DSQL.

--
WBR, SD.

[Dalton Calford]

I ended up rewriting all the utilities and removing the need for GPRE.   It may come back and bite me later, but we shall see.

I have made a linux and windows build and they are in the release sub-directory.

I only made 64bit releases but I am setting up cross compile build chains.

This is VERY early code and I have not even run the test scripts on it (I had to get ready for and participate in a job interview - hopefully I will get it)

What I would like, if people are willing, are testers who tell me the troubles they get setting it up and running it.

As always, all the code is in the repo. 

Thanks all

--
Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate
---
You received this message because you are subscribed to the Google Groups "firebird-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-deve...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/ffb539a3-6b60-4772-9d7d-bbd454a70336%40ibphoenix.com.

[Dimitry Sibiryakov]

Dalton Calford wrote 17 июл. 2025 г., 03:12:29:
> I ended up rewriting all the utilities and removing the need for GPRE.

I would like to see this commit.

--
WBR, SD.

[Dalton Calford]

It's already committed, look under utilities/modern in the src tree.

I already made a Alpha release for Windows and Linux.   I not working on driver's until the next cycle, but, you do have the API and sb_isql to play with.

Regards 

--
Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate
---
You received this message because you are subscribed to the Google Groups "firebird-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-deve...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/2f5b932b-dae0-4f3f-a095-1d8efa436ca4%40ibphoenix.com.

[Dimitry Sibiryakov]

Dalton Calford wrote 18.07.2025 15:49:
> It's already committed, look under utilities/modern in the src tree.

I see no code that perform any query there, so what code replaced GDML?

--
WBR, SD.

[Dalton Calford]

Damn, sorry, I was tired and posted an earlier bit of stub code, I will get the full thing in soon.   I was working on moving the utilities out of modern and into utilities while removing the old code and got the wrong stuff put in place - you missing the shared database header and support material, thanks for finding this.

Although it is a fun project, I get distracted at times and make mistakes - sorry about that.   I will post it up when I get back from a meeting this afternoon.

--
Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate
---
You received this message because you are subscribed to the Google Groups "firebird-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-deve...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/572c0bc4-6adf-4cab-9161-fff368737480%40ibphoenix.com.

[Dalton Calford]

Travelling on train while writing , expect auto-correct errors.

Writing this to distract my mind before meeting.

The problem happened due to one of the pitfalls of AI, it may have problem with code, so creates stub but not finished

It can even erase the code, it may create multiple different named versions of same code.

When working, I don't do it in main tree, I have a copy separate from main as a scratch code for the scratch code.

With ai it may try to reinvent wheel ignoring existing code.

I am making multiple changes to the underlying parser.

Instead of having to keep multiple parsers I am rescuing the already finished server code.

A change in the core doesn't mean I have to recreate clients, don't know if this is good or not, not really tested yet.

Like I said, this is scratch code in many ways.  I can run sql tests and verification against server, but, have not figured out way to fully test utilities - suggestions welcome.

hope this helpful

I will apply and post once I get home.

[Dalton Calford]

You should see the proper files in place for now - I have not reviewed the documentation, it compiles and the old GPRE files are moved into archive.  Other than that, this is not tested, have fun.

--
Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate
---
You received this message because you are subscribed to the Google Groups "firebird-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-deve...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/572c0bc4-6adf-4cab-9161-fff368737480%40ibphoenix.com.

[Dimitry Sibiryakov]

Dalton Calford wrote 20.07.2025 4:43:
> You should see the proper files in place for now - I have not reviewed the
> documentation, it compiles and the old GPRE files are moved into archive.

Using of ISC API is a big step back comparing to current Firebird sources.
Besides, I see that GDML is still used in the engine so GPRE is required for build.

--
WBR, SD.

[Dalton Calford]

It was specifically for the build, to get it out the door.  Also as a side note my instructions to the ai were "using the existing code as a template,  could you create a new version of the program that has all the functionality but does not use gpre"   all the "enhanced " crap was the ai doing what ai does.  I didn't even ask it to start.  I had to wait for it to pause before I could get it to save its state and stop trying to rewrite the engine inside of the utilities. 

I am still doing things in the core so the side utilities are more of an afterthought used to run tests against the new core.

I never said I was not going to eventually use gpre, just that I found it brittle and I did not want to keep teaching the ai how to compile it.

The biggest issue with ai is that it forgets what it is doing mid process and starts to add features.    This is due to the size of the context window. 

It also says it did something when it didn't or forgets that something is completed and starts to reimplement it. 

You will see markdown files throughout the code, these are used to try to keep the ai on track between memory compressions.

Now you see why I am doing this totally outside of the firebird project. 

If anyone likes something in it, they will have to do a complete review of the code in order to incorporate it. 

At the end of this,  I will have a great test bed as well as being better at using ai in real world use.

I can't wait to see what it does with the drivers and flamerobin.

All I say is,  don't look at the code for better ways of doing things but as inspiration. 

Best regards 

Dalton 

--
Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate
---
You received this message because you are subscribed to the Google Groups "firebird-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-deve...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/f752e454-6378-4efd-9877-ea4e314fdc82%40ibphoenix.com.

[Dalton Calford]

For those who are curious, do you want me to explain the steps I take in order to keep some rails on the AI?

For example, before I add a new feature or request a modification I run something like the following

"review the core server code base, in detail, list the features found in the code base, not those detailed in the documentation, we are looking for completed  
 features that are not marked complete or features that are marked complete but are not.  Don't bother with the utilities - only review the core server.  Put the  
 results into a file called CODEREVIEW_PRE6.md"

This is because I have seen modifying one part will have the AI remark out or remove a different section.

The prompt I gave above will actually have the AI run sb_isql to verify the feature works.  The issue is, just because it compiles and passes the first bit of testing, does not mean it works.

If you want I can keep giving feedback like this or just let the thread die

Let me know

[Dalton Calford]

I have been a bit distracted, but, over the week I have had the ai build documentation by reading the source code.  I am updating the build scripts and should have them done tomorrow.   For each element of the documentation I have instructed the AI to put links to the source code files where the feature is implemented.

So if you want to see what I have been working on, how it is used with links to the files it is implemented in, go to https://github.com/DaltonCalford/ScratchBird/blob/main/doc/INDEX.md

The ai tends to say "Production Ready" when it simply compiles and passes a few tests.   This is NOT production ready, but, it does have many features that may be interesting, that are implemented and have passed preliminary testing.

Let me know what you think.

[Pavel Cisar]

Dalton,

Dne 27. 07. 25 v 19:50 Dalton Calford napsal(a):

> I have been a bit distracted, but, over the week I have had the ai build
> documentation by reading the source code.  I am updating the build
> scripts and should have them done tomorrow.   For each element of the
> documentation I have instructed the AI to put links to the source code
> files where the feature is implemented.

It's impressive what the AI did (correctness aside, as I haven't had
time to verify it, but it looks ok at cursory glance).

It would be IMHO helpful if such documentation - with references to
source code - would be also available for Firebird main repository. In
fact, it might be interesting to have it done for each branch, including
development, so people could see where and how new things are
implemented as the engine evolves between versions. I can imagine that
in future such auto-documentation could be updated as part of CI
pipeline. Can github copilot do something like that?

best regards
Pavel

[Dimitry Sibiryakov]

Dalton Calford wrote 27.07.2025 19:50:
> This is NOT production ready, but, it does have many features that may be
> interesting, that are implemented and have passed preliminary testing.

It is not hard to "pass preliminary testing" when all that test utilities do
is print out "tests passed".

--
WBR, SD.

[Dalton Calford]

   It is not hard to "pass preliminary testing" when all that test utilities do
is print out "tests passed".

The test files in the tests directory are stubs, not the actual tests - the actual tests are all isql scripts - I have not included them in the source tree as I run them off my master tree that I play in, vs the one that I push up to github.

I am planning on putting all the tests as part of the ScratchRobin ui.  It will have a series of test scripts for both testing utility and for testing performance.  I will also have the tests included with each driver set as a driver verification pack.

That is after I get the next series of items implemented - finish documentation (what you see is about half of what has been generated) get the page size variations tested with all the new additions, get the Task manager created, get the replication servers created (three variations on replication available) and a whole series of small changes.   

I figure I have a few months of work to get it to the state I would want it to be.

For example, there are supposed to be three ways to link to a remote database ie the old way (execute on remote....) plus two new ways.

The first new way is a fixed link off of the Links schema (I am currently referring to these as hard links as they are permanent until dropped).   ie  links.server1.remoteschema1..... this uses optimizations such as a trusted authentication if both servers are sharing the same security database.  It also allows for a single connection to share multiple users - you can also say that all connections to the remote server act as a specific user.  You effectively are mounting a remote schema tree onto the local tree.  There are more things to its, but that is the basic concept.

The second new way is a direct link where you start your transaction, define the link to a specific table and give it a local name - ie select * from table@foo - and it only exists during the transaction.

I have at least a month of work/testing to get it stable enough to deal with network issues and other such problems.

It would be IMHO helpful if such documentation - with references to
source code - would be also available for Firebird main repository. In
fact, it might be interesting to have it done for each branch, including
development, so people could see where and how new things are
implemented as the engine evolves between versions. I can imagine that
in future such auto-documentation could be updated as part of CI
pipeline. Can github copilot do something like that?

 It was very difficult to get it to this state - it took four different ai's working in conjunction to get what you see produced.

AI's tend to kiss your backside when describing stuff - no matter how much I tell them not to, they start to run out of context window mid process and the backside kissing reappears.

It keeps saying things like "production ready" or "better than database x".

For example, https://github.com/DaltonCalford/ScratchBird/blob/main/doc/CONCLUSION.md
That document was filled with the "conclusions" put at the bottom of every document it processed.

I specifically told it not to put conclusions in - it ignored me until I told it to put all conclusions into a master conclusions document.

At this point, I would not say that ai could automatically generate this - but, if it is guided, it can generate it and then, it can maintain it once generated.

Figure about 30 hours of dev time by someone who understands AI to get it to produce what you want to generate the original docs.

Anyways, the main reason I included the implementation source files is that it helps me see what the AI has changed, as well as acting as a guide for new AI sessions to know where to look for something without needing to rescan the entire source tree

But, as a side benefit - it lets you guys go in and see what the AI did in implementing the code for a specific implementation.

As always, this is meant to be a learning package - I would love if some of the things implement become an inspiration for the Dev's in the main project

Best regards

Dalton

--
Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate
---
You received this message because you are subscribed to the Google Groups "firebird-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-deve...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/77a72685-b80d-409c-88d2-bb75fc02c096%40ibphoenix.com.

[Dimitry Sibiryakov]

Dalton Calford wrote 27.07.2025 22:47:
>    It is not hard to "pass preliminary testing" when all that test utilities do
> is print out "tests passed".
>
>
> The test files in the tests directory are stubs, not the actual tests - the
> actual tests are all isql scripts - I have not included them in the source tree
> as I run them off my master tree that I play in, vs the one that I push up to
> github.

In this case you also had to test something that is not included into the
tree as well because a quick glance reveals that actual implementation of
described features is missing too.


--
WBR, SD.

[Dalton Calford]

Which features are you referring to?

It's quite possible that a file may be corrupted as AI will delete things to get something to compile, but, my last build and test scripts passed, so if something is missing on the push, I need to find out what and why

--
Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate
---
You received this message because you are subscribed to the Google Groups "firebird-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-deve...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/f3f1a418-c945-4682-a437-79922cbe8478%40ibphoenix.com.

[Dimitry Sibiryakov]

Dalton Calford wrote 27.07.2025 23:55:
> Which features are you referring to?

Specifically I looked for array functions and bitmap indexes. None there.

--
WBR, SD.

[Dalton Calford]

ok, I will look in the morning - my sunday night pathfinder game just started

--
Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate
---
You received this message because you are subscribed to the Google Groups "firebird-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-deve...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/firebird-devel/03f52ba1-43fa-47ea-badb-637495b285cc%40ibphoenix.com.

[Mark Rotteveel]

I am pretty much anti-AI, and I don't see the point of having the
hallucinations and bullshit spewed by it - at great environmental cost I
might add - included in the project.

Mark
--
Mark Rotteveel

[Dimitry Sibiryakov]

'Mark Rotteveel' via firebird-devel wrote 28.07.2025 9:49:
>> It would be IMHO helpful if such documentation - with references to source
>> code - would be also available for Firebird main repository. In fact, it might
>> be interesting to have it done for each branch, including development, so
>> people could see where and how new things are implemented as the engine
>> evolves between versions. I can imagine that in future such auto-documentation
>> could be updated as part of CI pipeline. Can github copilot do something like
>> that?
> I am pretty much anti-AI, and I don't see the point of having the hallucinations
> and bullshit spewed by it - at great environmental cost I might add - included
> in the project.

Perhaps instead of recreation of Language Reference it worth to try to use AI
to create something that is really needed: OO API Reference.

--
WBR, SD.

[Mark Rotteveel]

And have documentation full of hallucinations of methods that don't
exist, or usage instructions that are entirely wrong, and requiring
someone to _verify_ everything which can be a lot more work than just
writing the damn thing.

Mark
--
Mark Rotteveel

[Dimitry Sibiryakov]

'Mark Rotteveel' via firebird-devel wrote 28.07.2025 10:37:
>
> And have documentation full of hallucinations of methods that don't exist, or
> usage instructions that are entirely wrong, and requiring someone to _verify_
> everything which can be a lot more work than just writing the damn thing.

Most likely yes, but if the AI-generated draft is put on the Wiki with a big
warning, this work can be distributed among casual editors who are normally
unable to create the damn thing at all.

--
WBR, SD.

[Mark Rotteveel]

I think the history of our documentation pretty much proves that such a
thing is not going to happen.

Mark
--
Mark Rotteveel

[Dalton Calford]

1. 
   /src
2. /jrd

/

BitmapIndex.cpp 

 look around line 539 for the bitmap index (there are a series of Bitmap files in the jrd directory) 
array work is spread across multiple files 

For the system functions, I found that when I had the ai move records in a sub-agent, I did not give the sub-agent enough output tokens (yes, there are a max input and output tokens that need to be set for each ai and its sub-agents)

So the file was "summarized" vs being moved/adjusted as needed.

I am going to work on this when I get home tonight.

Thankfully the original source is in my other tree.

This is one of the reasons you have to be very careful and this is part of the learning curve I am going through.

[Dimitry Sibiryakov]

Dalton Calford wrote 28.07.2025 18:25:
>  look around line 539 for the bitmap index (there are a series of Bitmap files
> in the jrd directory) array work is spread across multiple files

Look around line 942. This is an in-memory sparse bitmap structure which has
no relation to a database.

--
WBR, SD.