Code 6 Tuning

[Henrietta Naughton]

Codetuning is appealing for several reasons. One attraction is that it seems to defy the laws of nature. It's incredibly satisfying to take a routine that executes in 20 microseconds, tweak a few lines, and reduce the execution speed to 2 microseconds.

In practice, there is not such a big difference in running timebetween an algorithm with growth rate \(\Theta(n)\) and anotherwith growth rate \(\Theta(n \log n)\).There is, however, an enormous difference in running time betweenalgorithms with growth rates of \(\Theta(n \log n)\) and\(\Theta(n^2)\).As you shall see during the course of your study of common datastructures and algorithms, there are many problemswhose obvious solution requires \(\Theta(n^2)\) time,but that also have a solution requiring \(\Theta(n \log n)\)time.Examples include sorting and searching, two of the most importantcomputer problems.

Here are some suggestions for ways to speed up yourprograms by code tuning.The most important thing to realize is that most statements in aprogram do not have much effect on the running time of that program.There are normally just a few key subroutines, possibly even keylines of code within the key subroutines, that account for most ofthe running time.There is little point to cutting in half the running time of asubroutine that accounts for only 1% of the total running time.Focus your attention on those parts of the program that have the mostimpact.

When tuning code, it is important to gather good timing statistics.Many compilers andoperating systemsinclude profilers and other special tools to help gather informationon both time and space use.These are invaluable when trying to make a program more efficient,because they can tell you where to invest your effort.

A lot of code tuning is based on the principle of avoiding work ratherthan speeding up work.A common situation occurs when we can test for a condition that letsus skip some work.However, such a test is never completely free.Care must be taken that the cost of the test does not exceed theamount of work saved.While one test might be cheaper than the work potentially saved, thetest must always be made and the work can be avoided only somefraction of the time.

A common operation in computer graphics applications is to findwhich among a set of complex objects contains a given point inspace.Many useful data structures and algorithms have been developed todeal with variations of this problem.Most such implementations involve the following tuning step.Directly testing whether a given complex object contains the pointin question is relatively expensive.Instead, we can screen for whether the point is contained within abounding box for the object.The bounding box is simply the smallest rectangle (usually definedto have sides perpendicular to the \(x\) and \(y\) axes)that contains the object.If the point is not in the bounding box, then it cannot be in theobject.If the point is in the bounding box, only then would we conduct thefull comparison of the object versus the point.Note that if the point is outside the bounding box, we saved timebecause the bounding box test is cheaper than the comparison of thefull object versus the point.But if the point is inside the bounding box, then that test isredundant because we still have to compare the point against theobject.Typically the amount of work avoided by making this test is greaterthan the cost of making the test on every object.

Asymptotic algorithm analysisis an analytic tool, whereby we model the key aspects of analgorithm to determine the growth rate of the algorithm as the inputsize grows.It has proved hugely practical, guiding developers to use moreefficient algorithms.But it is really an estimation technique, and it has itslimitations.These include the effects at small problem size, determining the finerdistinctions between algorithms with the same growth rate, andthe inherent difficulty of doing mathematical modeling for morecomplex problems.

An alternative to analytical approaches are empirical ones.The most obvious empirical approach is simply to run two competitorsand see which performs better.In this way we might overcome the deficiencies of analyticalapproaches.

Another approach to analytical analysis is simulation.The idea of simulation is to model the problem with a computer programand then run it to get a result.In the context of algorithm analysis, simulationis distinct from empirical comparison of two competitors because thepurpose of the simulation is to perform analysis thatmight otherwise be too difficult.A good example of this appears in the following figure.

This figure shows the cost for inserting or deleting a record from ahash table under two differentassumptions for the policy used to find a free slot in the table.The \(y\) axes is the cost in number of hash table slotsevaluated, and the \(x\) axes is the percentage of slots in thetable that are full.The mathematical equations for these curves can be determined,but this is not so easy.A reasonable alternative is to write simple variations on hashing.By timing the cost of the program for various loading conditions, itis not difficult to construct a plot similar to this one.The purpose of this analysis was not to determine which approach tohashing is most efficient, so we are not doing empirical comparison ofhashing alternatives.Instead, the purpose was to analyze the proper loading factor thatwould be used in an efficient hashing system to balance time costversus hash table size (space cost).

At the recent Fabric Conference, we announced that both code-first automated machine learning (AutoML) and hyperparameter tuning are now in Public Preview, a key step in making machine learning more complete and widely accessible in the Fabric Data Science.

Our system seamlessly integrates the open-source Fast Library for Automated Machine Learning & Tuning (FLAML), offering a tailored version directly within the default Fabric 1.2 runtime. This means that users can access both AutoML and Tune capabilities without the need for any extra installation or configuration steps.

In Fabric Data Science notebooks, users can tune their machine learning models using flaml.tune. This feature empowers users to meticulously search for the most effective hyperparameters, ensuring models reach their highest potential performance. With flaml.tune, users can navigate extensive hyperparameter spaces with ease, quickly pinpointing the best configurations for optimal outcomes.

Automated Machine Learning (AutoML) streamlines the development of machine learning models by automating training and optimization, eliminating the need for deep technical expertise. This capability simplifies the traditionally complex and time-consuming processes of selecting algorithms, tuning hyperparameters, and validating models. This innovation democratizes machine learning, making advanced data analysis accessible to both experts and novices across various industries to solve complex problems and drive innovation.

With AutoML, users can take their training data and a provide a machine learning task to find the best model. The integration of flaml.automl into Fabric takes advantage of Apache Spark to enable AutoML at scale, providing users with capabilities such as:

Begin your journey with hyperparameter tuning and AutoML directly from the Fabric Data Science homepage by exploring the AI Samples gallery. These tutorials guide you through utilizing AutoML and Tune within Fabric Notebooks, streamlining the process to efficiently optimize and develop your ML models.

One thought I have is to make sure that every small script (or function of a large script) has a docstring. The tuning would then be prompt = docstring, response = code. This is a pain if the existing code base is not well commented. I did wonder about asking GPT to summarise code chunks (small scripts, functions in large scripts) and then feeding these as prompts into fine tuning.

@Foxalabs I do accept your point. GPT-4 is much better at coding than GPT-3 so a well tuned GPT-3 might not match GPT-4. On the other hand, if tuning on GPT-4 arrives long before GPT-5 or whatever, then it could be of interest.

I offer completion examples, increasing from creative writing completions to elucidating answers, to get you thinking about how to interact with the models without putting another layer of difficulty in the task of fine-tuning:

However, to me, the form of prompting is tweakable or even redundant there. The system message would seem to inspire what comes after your pseudocode in python code form, rather than rewriting as python. davinci-003 follows it reasonably well, though - up until that alternate token gets picked as the first.

User: write python: banana_farm function is defined. A message is printed with the purpose of the banana farm game. Then we ask how many players. Then we introduce the rules. Then initialize the variables needed to maintain the game state.

Code models can be tuned by usingsupervised tuning.Supervised tuning uses labeled examples that demonstrate the type of outputyou'd like from your code generation or code chat model during inference. Codemodels don't support tuning by usingReinforcement learning from human feedback (RLHF) tuning.

Tuning is required when you want a model to learn something niche or specificthat deviates from general language and code patterns. The following areexamples of what you can teach the code-bison and codechat-bison models:

The dataset used to tune a code model includes examples that align withthe task that you want the model to perform. Your dataset must include a minimumof 10 examples, but we recommend at least 500 examples for good results.The more examples you give, the better the results.

Your code generation model tuning dataset must be in JSONLines (JSONL) format where each linecontains a single tuning example. Each example is composed of aninput_text field that contains the prompt to the model and anoutput_text field that contains an example response that the tuned modelis expected to produce.

3a8082e126