Do you have complete visibility into how your compiler handles the latest language specifications?
SuperTest Compiler Test and Validation Suite
SuperTest is the test and validation suite for C and C++ compilers and libraries that has tracked the (ISO) language specifications for more than 30 years. It is a complete validation environment with
A huge collection of hand-written test- A large body of generated tests
- A configurable test driver to easily set-up and run validations
- A report generator that provides comprehensive overviews of validation results in multiple formats, including HTML and JUnit XML, and will immediately identify regressions of the latest compiler and library version compared to a designated reference
Verification of Diagnostics
There are quite some cases where the language specification requires the compiler to issue a diagnostic. SuperTest contains an elaborate subset of tests that must fail at compile-time. If they don’t, SuperTest will tell you about it.
Code Size Tracking
SuperTest’s Code Size Tracking is an easy way to monitor changes in generated code size between compilers or between different compiler settings. Code Size Tracking is built into SuperTest’s test-driver
Calling Convention Tester
The calling convention of a compiler specifies how functions receive parameters from the functions that called them and how their result is returned to the calling function. This implementation can be very complex, with the goal to make the action of calling a function as efficient as possible.
The SuperTest suite covers the following ISO specifications
- ISO/IEC 9899:1990 C90 standard
- ISO/IEC 9899:1999 C99 standard
- ISO/IEC 9899:2011 C11 standard
- ISO/IEC 9899:2018 C18 standard
- ISO/IEC TR 18037:2008 Embedded C extension
- ISO/IEC 14882:2003 C++03
- ISO/IEC 14882:2011 C++11
- ISO/IEC 14882:2014 C++14
- ISO/IEC 14882:2017 C++17
- MISRA C:2004
- MISRA C:2012
- MISRA C:2012 Revision 1
The test suite is systematically structured so that it is clear what section of which standard is tested by each test. Therefore, it is also easy to configure which language version should be tested when executing the test suite.
Elevate, Optimize, and Secure with Meteonic Innovation
Our Mobile Application Security Testing Tools offer comprehensive solutions to safeguard your mobile applications against evolving cyber threats. From code obfuscation to anti-tampering features, our tools provide multi-layered defense to protect your apps and user data.
Standard Library Tests
While SuperTest is traditionally positioned as a tool for compiler validation, it is important to understand that the C and C++ language specifications define both the compiler language and the standard library. SuperTest validates both! The library suite of SuperTest covers C90, C99, C11, C18, C++03, C++11, C++14 and C++17.
The difference between the compiler language and standard library is important in the light of qualification for safety critical applications. The compiler handles the implementation of the language, and it is the tool that generates the target code. SuperTest is a perfect tool for compiler qualification. For standard library qualification you need more than the library tests suite in SuperTest. The library consists of code that is linked to the application and is actually loaded into the device. For use in safety critical applications it is needed to provide requirement level traceability from the ISO language specification to the individual tests. To do so, we offer the SuperGuard C Library Safety Qualification Suite as an add-on to SuperTest.
Optimization Coverage
Testing the optimizers is one of the hardest tasks in compiler validation. We have extended our test suite to maximize source code coverage of compiler optimizers, thereby maximizing the test-coverage of your compiler. We developed a test suite that specifically focuses on loop optimizations. Loop optimizations, which transform the program’s control graph significantly, require the additional verification techniques that are included in this optimization suite. For example, the tests in this test suite are designed to execute all of the code and branches that are generated by the compiler at the assembly code level, such that there are no gaps in the test coverage of the optimized program. Verification of Optimization Correctness with SuperTest
Easy to Use for Any Compiler and Target
SuperTest is easy to install on Linux, Mac OS X and Windows, 32 and 64-bit hosts. You are up and running in minutes. SuperTest’s straightforward scripts for compilation and execution make it easy to adapt to the most complicated compiler and execution environment setup. Do you have a simulator? No problem. Do you have a processor board at the other end of an RS-232 cable? No problem. SuperTest can be configured to qualify any C or C++ compiler toolchain for any execution environment and we provide example configurations for many of them.
New SuperTest features over the years
Trust in SuperTest's 30+ years of refinement and innovation
When SuperTest started, there was only K&R C. As ISO standardization progressed, SuperTest was extended to track the latest developments. We plugged holes in the suite by adding tests for errors not initially found. We created generators to add systematic testing for complex parameter spaces. SuperTest is now the most complete validation suite for C and C++ available
SuperTest includes many more unique features that are not found in other test suites
Our Unique World Class DevSecOps Solutions Enables You to Implement Shift Left and Get You Desired Benefits
- Very easy and quick installation and configuration
-
- Can be used to verify deeply embedded freestanding C implementations on memory constraint devices (down to 4K memory)
-
- Clear and easy reporting in text, HTML, JUnit XML and CSV
-
- The CGtrainer (Code Generator trainer), a suite of nearly one-thousand tests aimed at boot-strapping new compiler development
-
- The ABI-tester, which can be used to verify calling conventions within the compiler, between different versions of the compiler, or even between completely different compilers
-
- Powerful testing of the code generator/back-end
-
- The Depth-Suite is a collection of generated tests that exhaustively verifies arithmetic correctness relative to the target machine’s data model. Depth suites for many data models are included, but if your data model is not there, we will create one for you free of charge
-
- Parallel testing automatically splits validation runs into a preconfigured numbers of threads to speed up the validation
-
- Testing of libraries
-
- A production-rule driven test generator to create pseudo-random variations of a particular test
-
- Easy addition of your own specific tests
-
- Excellent professional support
