Sss R3 File Generator | Tested
Casandro Diveley
I have developed a code-generator that takes our python interface to our C++ code (generated via SWIG) and generates code needed to expose this as WebServices. When I developed this code I did it using TDD, but I've found my tests to be brittle as hell. Because each test essentially wanted to verify that for a given bit of input code (which happens to be a C++ header) I'd get a given bit of outputted code I wrote a small engine that reads test definitions from XML input files and generates test cases from these expectations.
So I'm trying to think of alternative approaches to this problem, and it strikes me I'm perhaps tackling it the wrong way. Maybe I need to focus more on the outcome, IE: does the code I generate actually run and do what I want it to, rather than, does the code look the way I want it to.
I started writing up a summary of my experience with my own code generator, then went back and re-read your question and found you had already touched upon the same issues yourself, focus on the execution results instead of the code layout/look.
Recall that "unit testing" is only one kind of testing. You should be able to unit test the internal pieces of your code generator. What you're really looking at here is system level testing (a.k.a. regression testing). It's not just semantics... there are different mindsets, approaches, expectations, etc. It's certainly more work, but you probably need to bite the bullet and set up an end-to-end regression test suite: fixed C++ files -> SWIG interfaces -> python modules -> known output. You really want to check the known input (fixed C++ code) against expected output (what comes out of the final Python program). Checking the code generator results directly would be like diffing object files...
If you are running on *nux you might consider dumping the unittest framework in favor of a bash script or makefile. on windows you might consider building a shell app/function that runs the generator and then uses the code (as another process) and unittest that.
A third option would be to generate the code and then build an app from it that includes nothing but a unittest. Again you would need a shell script or whatnot to run this for each input. As to how to encode the expected behavior, it occurs to me that it could be done in much the same way as you would for the C++ code just using the generated interface rather than the C++ one.
Provided you have your generated code assembled from smaller chunks, and the chunks do not change frequently, you can exercise more conditions and test a little better, and hopefully avoid having all your tests break when you change specifics of one chunk.
So in conclusion, a single unit / specification can have multiple behaviours, and the specification grows as you develop the unit / system; and if your system under test depends on other concrete systems within it, watch out.
My recommendation would be to figure out a set of known input-output results, such as some simpler cases that you already have in place, and unit test the code that is produced. It's entirely possible that as you change the generator that the exact string that is produced may be slightly different... but what you really care is whether it is interpreted in the same way. Thus, if you test the results as you would test that code if it were your feature, you will find out if it succeeds in the ways you want.
Basically, what you really want to know is whether your generator will produce what you expect without physically testing every possible combination (also: impossible). By ensuring that your generator is consistent in the ways you expect, you can feel better that the generator will succeed in ever-more-complex situations.
In this way, you can also build up a suite of regression tests (unit tests that need to keep working correctly). This will help you make sure that changes to your generator aren't breaking other forms of code. When you encounter a bug that your unit tests didn't catch, you may want to include it to prevent similar breakage.
In my case, the program generates many types of code (C#, HTML, SCSS, JS, etc.) that compile into a web application. The best way I've found to reduce regression bugs overall is to test the web application itself, not by testing the generator.
Have you ever asked how often should a generator be load tested? During times of an outage, many companies must have a reliable standby generator in order to keep their essential services operating. Something as simple as a power outage can determine life and death situations for hospitals and clinics. Fortunes can be lost for those working in the financial sector during downtime.
For these situations, and many others like them, it is crucial for companies to test their emergency backup generators periodically, ensuring everything is in working order for when they are actually needed the most. This process is called load bank testing.
A load bank is a piece of equipment that is specialized to produce an electrical load. It then applies that load to a generator by bringing the engine up to a particular operating pressure and temperature to simulate the equipment that is meant to be used during a power outage emergency.
One of the many benefits of load bank testing is the peace of mind you will get knowing that your generator will work properly during an emergency. Another benefit is the amount of money you can save by catching any faults with the system during testing, rather than during an emergency when time costs money.
The National Fire Protection Agency, or the NFPA, has set standardized guidelines that organizations and businesses must follow. Businesses with potentially critical situations will follow the NFPA 110 guidelines, while organizations with less critical situations follow the NFPA 70 guidelines.
Depending upon the purpose of the generator and how it is classified, the schedule for load bank testing your generator will vary. For example, the testing for crucial services such as hospitals and clinics where there is the possibility of serious injury or even loss of life is more frequent than that of a grocery store. Financial organizations and data centers follow the stronger NFPA guidelines, as well.
For most businesses, the NFPA recommends that all non-mission critical generators should be tested from 30% to 50% of their maximum load once a month for thirty minutes each time. Diesel generators in potentially critical situations can be tested more frequently. However, this is dependent on industry regulations and individual requirements.
For businesses that operate several generators, it is more advantageous to have a load bank stationed on-site permanently. However, other smaller businesses may wish to rent a load bank tester rather than buying.
Following a checklist during the testing procedures is essential for maintaining safety for the persons completing the test. A checklist will also help ensure all steps are followed during the procedure protecting the generator from a complete power failure during the test.
Before conducting the test, the tester should put the appropriate safety clothing on. This includes wearing head, hearing, and eye protection, as well as gloves, steel-tipped boots, and a high visibility vest.
Prime Power Rentals offers state-of-the-art generator rental equipment and gold-standard turnkey services throughout New England and New York. We are a 24/7/365 emergency generator rental provider with a 1-hour response time, complete setup and breakdown packages, and on-site management.
Trying to schedule a transformer replacement or performing system upgrades, but need to minimize outage duration? Whatever the application, Prime Power Rentals can provide the tools you need to help minimize customer downtime.
Middle of the night downed pole due to a vehicle accident? Pad mount transformer failure serving an assisted living center? Emergency power interruptions happen every day due to a number of reasons ranging from inclement weather to an aging infrastructure.
This is a HUGE liability. Personally I would refuse. I would site that the cost of renting an generator, hiring an electrician and hooking it up properly costs more than having the electricity turned on. Not to mention the liability.
In general, you should conduct no-load testing on a weekly or monthly basis. Load testing should occur at least monthly or quarterly. Automating the process ensures your generators receive regular exercise and helps you maintain the appropriate schedule.
Do you operate a business in Nebraska or Pottawattamie County, Iowa? Your nearest NMC Cat can help keep your generators in prime condition. Our experienced power systems technicians can perform regular testing and maintenance that fits your time frame and budget.
We also use third-party cookies that help us analyze how you use this website, store your preferences, and provide the content and advertisements that are relevant to you. These cookies will only be stored in your browser with your prior consent.
Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.
We recently had to update our customer instances and used the ATF test generator and cloud runner for testing. It allowed us to quickly set up tests for new features and run them from multiple locations simultaneously. The real-time results helped us spot and fix issues instantly, ensuring a smooth experience for their B2C.
About 2 months ago we started exploring ATF test generator and cloud runner. You can get it up and running pretty quickly, do be aware that it won't work immediately, you will see a prompt that it can take 24 hours because ServiceNow needs to configure something (took for us about half an hour).
7fc3f7cf58