One to rule them all
The world of software development is built on dreams of engineers who aspired to create their own programming languages. While some succeeded, most of these languages never reached any broader audience and even those who did, mostly faded into obscurity after a decade or two. Programming languages often emerge and disappear as technology evolves and only a handful, like C, have stood the test of time, remaining relevant and widely used.
Like many before me, I’m haunted by the thought of a better way to create software, especially when I grapple with the absurdities of existing languages. I’ve worked mostly with C, C++, Python, CMake and Bazel, so if you know any of these, I’m sure you realize that each of them has a wide range of absurdities to offer.
Yet, whenever someone declares, “I’m going to create a new universal programming language”, I’m very skeptical. My mind immediately fires off questions like:
- Do you understand the immense effort required to design a new programming language?
- Do you realize the work involved in implementing it?
- Are you aware of how many have tried and failed before you?
- Do you comprehend the vast tooling ecosystem needed to support a professional-grade language?
- Will making your programming language really be more efficient than forcing one of the existing languages to do what you need to do?
- Do you understand the limitations of current languages and the reasons behind them?
- Are you solving a real problem or are you simply driven by the desire to create something new?
But the most critical question is:
- Why would anyone choose your language over the established ones?
Failing to address the last question is, in my view, the root cause of many frustrating and pointless debates across programming forums and social media. Add to this equation excellent communication skills of an average software developer and you have a typical conversation under a “I’ve created a new programming language” post looking like this:
Author: I’ve been writing software in C++ for a year and I’m tired of typing variable types. So, I’m designing a new language where you don’t have to specify types!
Responder: Wtf? Python has that. Basically all the dynamically typed languages have that. Even better – C++, which you claim to use, has auto keyword, so you don’t need explicit type names in many cases.
Author: After your response I’ve read about that dynamically typed language thingy, but we don’t use them in our project. And as for the auto keyword, it requires new C++11, which we don’t use. My language would let engineers skip type declarations without upgrading to C++11!
Responder: “It’s 2024 – C++11 is 13 years old! Do you really think it’s easier for people to switch to an unknown language maintained by one person than to upgrade to a newer C++ standard, especially if they’re already using C++? Author: “You don’t get it! For those interested in my language, here’s the GitHub repo!”
You click the link and…you see that the project died 3 months after its creation. Maybe the author learned more about that “dynamically typed thingy”. Perhaps their project finally adopted C++11. Or maybe the author realized that one year of experience isn’t enough to develop a viable programming language. Most likely, though, the motivation to create a new language was more about feeding an engineer’s ego than solving a real problem.
Since we’re talking about that – the engineer’s ego is a powerful force. Engineers often lose sight of the bigger picture. There’s a huge world out there that just wants to have a car navigation or a seamless mobile banking experience. That world doesn’t care about C++, Python, Jenkins, or REST APIs. But engineers?
An engineer is often a person who just really wants to write something in Python. Or really wants to set up CI with Jenkins. Not because it’s the best solution for the problem, but because it’s what they prefer. This is a dangerous mindset, as it can lead to endless technical debates that overlook the actual needs the solution is supposed to address.
Take, for example, the old debate about C++’s backward compatibility with C. In my opinion, preserving that compatibility was one of the best decisions the C++ designers made. Why? Because it leveraged humanity’s most powerful approach – building on what already exists. By connecting C and C++, engineers could transition gradually from the old to the new without starting from scratch. No library for your new C++ project? Use an old C library — it’ll work. Don’t yet know how to do something in C++? Write it in C. This compatibility lowered the entry barrier for C developers moving to C++. And of course that it brought challenges and difficult compromises visible till this day in the mixed C/C++ code, but let’s be serious – there are no perfect solutions. Specifically, there are no perfect solution on the engineering side because the source code exists for the business, not vice versa. That means that if there have to be any compromise, it will most likely be on the engineering side, not on the business side.
What’s interesting is that failures in software development are not limited to individual efforts. Even large-scale projects often fall short. How many times have we heard promises of a unified package manager for C++ like Conan or a do-it-all language like Python? Many projects start with “unification” in mind, but once they reach a broader audience, it becomes clear that no solution fits all scenarios. Even such a great success story like Python. People have done huge amount of work to allow using Python everywhere, even in the embedded systems, but still areas like world of embedded software is conquered by unsafe, inconvenient and hated C and C++.
What is this series about?
C++ has now been around for almost 40 years and while new programming languages more suited to modern software development have emerged, C++ continues to evolve with updated versions. However, in comparison to these newer languages, C++ has begun to resemble an intermediate form rather than a clean input source code which is convenient to write by people. Despite the advantages of the newer programming languages, the reality is that there are (and will be) vast C++ code bases that will require maintenance and development, much like the situation with C.
So, what am I doing here? I would like to research two areas described below.
What can I use to write C++ code for the existing code bases more efficiently?
This scenario represents the situation in which I must add a C++ code to an existing code base which is written and maintained entirely in C++. However, I don’t want to write it in C++ simply because of how verbose and complex the language is – there’s just too much typing which takes too much time and there are too many pitfalls which should be handled by the tool, but they are not.
Some tool --> C++ source code --> Existing C++ projectWhat can I use to write new projects and still be able to utilize the existing C++ code?
This scenario represents the situation in which the implementation of some utils already exists in C++, but the new project can be written in a different language.
Existing C++ code --> New non-C++ software projectHow to compare different tools and solutions?
To make that happen, I need some sort of a reference code which I could treat like a benchmark when researching other solutions. Such code must be as short as possible, but at the same time it must be representative, meaning that it must contain many language features, so that then I can compare these features between different languages.
After some time, I ended up with the following reference code:
#include <iostream> // needed for std::cout
#include <memory> // needed for std::shared_ptr
#include <shared_mutex> // needed for std::shared_mutex
#include <chrono> // needed for time measurement
#include <mutex> // needed for std::unique_lock
#include <vector> // needed for std::vector
#include <condition_variable> // needed for std::condition_variable_any
#include <array> // needed for std::array
#include <future> // needed for std::async
#include <atomic> // needed for std::atomic
// mutex for shared resource access synchronization
static std::shared_mutex mtx;
// condition variable for threads execution flow synchronization
static std::condition_variable_any cv;
// atomic variable to share a flag between the threads
static std::atomic<bool> is_finished = false;
// interface class
class WorkerInterface
{
public:
virtual ~WorkerInterface() = default;
virtual void Run() = 0;
};
// alias for a type
using WorkerPtr = std::shared_ptr<WorkerInterface>;
// templated alias for a type
template <typename T>
using SharedContainerPtr = std::shared_ptr<std::vector<T>>;
// template class definition + inheritance
template <typename T>
class Producer : public WorkerInterface
{
public:
// class constructor and members initialization
// together with passing arguments by const reference and non-const reference
Producer(const std::array<T, 3> &source_data, SharedContainerPtr<T> shared_container)
: source_data_{source_data},
shared_container_{shared_container}
{
// if statement
if (!shared_container_) {
// error reporting by throwing an exception
throw std::runtime_error("Given shared_container is a nullptr!");
}
}
void Run() override
{
// indexed for loop
for (unsigned int i=0U; i<10000U; i++) {
// range based for loop
for (const auto &element : source_data_) {
// print statement
std::cout << "Putting " << element << " to the shared container" << std::endl;
{
// blocking reading and writing from/to the shared container
std::unique_lock<std::shared_mutex> lock(mtx);
// appending element to a variable-length array
shared_container_->push_back(element);
}
// unblocking all waiting threads
cv.notify_all();
}
}
is_finished = true;
cv.notify_all();
std::cout << "Producer done" << std::endl;
}
// private class members
private:
// fixed-length array with a generic type
std::array<T, 3> source_data_;
SharedContainerPtr<T> shared_container_;
};
template<typename T>
class Consumer : public WorkerInterface
{
public:
Consumer(const unsigned int id, SharedContainerPtr<T> shared_container)
: id_{id},
last_size_{shared_container->size()},
shared_container_{shared_container}
{
if (!shared_container) {
throw std::runtime_error("Given shared_container is a nullptr!");
}
}
void Run() override
{
// infinite while loop
while (true) {
// blocking writing to the shared container
std::shared_lock<std::shared_mutex> lock(mtx);
// waiting for the input from another thread
cv.wait(lock, [this]{ return shared_container_->size() > last_size_ || is_finished; });
if (is_finished) {
break;
}
std::cout << "Consumer " << static_cast<unsigned>(id_) << " noticed new element: " << shared_container_->back() << std::endl;
last_size_ = shared_container_->size();
}
std::cout << "Consumer " << static_cast<unsigned>(id_) << " done" << std::endl;
}
private:
unsigned int id_;
std::size_t last_size_;
SharedContainerPtr<T> shared_container_;
};
// enum definition
enum class WorkerType
{
kProducer,
kConsumer
};
// template function with a compile time argument and variadic arguments
template <WorkerType worker_type, typename ... Args>
[[nodiscard]] WorkerPtr CreateWorker(Args&&... args)
{
// compile time if statement excluding constructors of the classes to which
// current arguments don't fit - an ordinary runtime if would cause a compilation error
// because Producer's and Consumer's constructors have different signatures
if constexpr (worker_type == WorkerType::kProducer) {
// dynamic memory allocation of the polymorphic type
return std::make_shared<Producer<std::string>>(std::forward<Args>(args)...);
}
else if constexpr (worker_type == WorkerType::kConsumer) {
return std::make_shared<Consumer<std::string>>(std::forward<Args>(args)...);
}
else {
throw std::runtime_error("Unsupported worker type: " + std::to_string(static_cast<unsigned int>(worker_type)));
}
}
// application entry point
int main(int argc, char** argv)
{
// start measuring execution time
const auto start = std::chrono::high_resolution_clock::now();
try {
// fixed-length array
const std::array<std::string, 3> source_data {"Hello", "world", "!!!"};
// variable-length array
std::vector<std::string> message_container;
SharedContainerPtr<std::string> shared_container = std::make_shared<std::vector<std::string>>(message_container);
// creation of the polymorphic types
WorkerPtr producer = CreateWorker<WorkerType::kProducer>(source_data, shared_container);
WorkerPtr first_consumer = CreateWorker<WorkerType::kConsumer>(0U, shared_container);
WorkerPtr second_consumer = CreateWorker<WorkerType::kConsumer>(1U, shared_container);
// start 3 threads
auto producer_future = std::async([producer](){ producer->Run(); });
auto first_consumer_future = std::async([first_consumer](){ first_consumer->Run(); });
auto second_consumer_future = std::async([second_consumer](){ second_consumer->Run(); });
}
// error handling
catch (const std::exception &e) {
std::cout << "Error: " << e.what() << std::endl;
}
// measure execution time and display the result
const auto end = std::chrono::high_resolution_clock::now();
std::cout << "Time elapsed: " << std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count() << "ms" << std::endl;
}The properties of this code that I’ll be interested in when comparing to other solutions are:
- length of the code – this code is 124 lines long (without comments and empty newlines, of course)
- build time – it takes 2149ms to compile a release version (median out of 20 compilation runs) with g++:
g++ producer_consumer.cpp -O3 -DNDEBUG -march=native -std=c++23 -o producer_consumer- execution time – it takes 597ms to run it (median out of 20 runs)
- binary size – 105kB
For testing the ability to reuse the existing C++ code, I will use the following code:
// cpp_functionality.h
#pragma once
// example C++ struct to import
struct Point
{
bool operator==(const Point& other) const
{
return x == other.x && y == other.y;
}
int x, y;
};
// ordinary function
int Add(const int a, const int b);
// template function
template <typename T>
bool IsEqual(const T a, const T b)
{
return a == b;
}// cpp_functionality.cpp
#include "cpp_functionality.h"
// Add implementation
int Add(const int a, const int b)
{
return a + b;
}// lib_user.cpp
#include <iostream>
#include "cpp_functionality.h"
int main(int argc, char** argv)
{
Point p1 {12, 24};
Point p2 {36, 48};
const int adding_result = Add(p1.x, p2.x);
const bool comparison_result = IsEqual(p1, p2);
std::cout << "adding_result = " << adding_result << std::endl;
std::cout << "comparison_result = " << comparison_result << std::endl;
}I showed the main function implementation in C++ just to present what behavior I’m going to achieve, but throughout this series it will be implemented in other languages.
You can download all the relevant source code (including the one above and all the future implementations in other languages) from this GitHub repository.
The goal and the structure
By making such research I want to give myself some new tools, work out approaches and methods which allow for faster, safer and more convenient software development. It is supposed to be an experiment during which I want to:
- prepare a reference C++ code snippets to use when analyzing other tools (the current article)
- define what “better” means in context of the software development process
- understand what are the existing alternatives meant to address C++ shortcomings
- check the modern ways of approaching the topic of building a new programming languages
The elephant in the room: new C++ versions
Shouldn’t we just wait for newer C++ versions and hope that the main issues will be fixed? Ideally, yes, but the problem isn’t the lack of updates – it’s the direction in which C++ is being developed. In my opinion, the last significant update that truly addressed core issues related to safety and ease of development was C++11. Since then, the C++ ISO committee seems to be more focused on expanding the language’s capabilities, but the versatility of C++ was never in question. It’s a powerful language that can achieve almost anything in software development, so judging by the features being added in recent updates, I’m skeptical that C++ will ever become what I would consider “a better place.”
Why start with C++?
You might wonder why I’m focusing on C++ so much, especially after criticizing engineers for being too tech-focused. The reason is simple: I work primarily in C++, so this research is meant to be useful mainly for C++ engineers. However, this doesn’t mean I’ll ignore other languages. If I’m serious about building on what we already have, I just can’t do that.
Next: What does better mean?
