Serializing access to Streams

Last december, I was at Meeting C++ in Berlin and I attended – among others – the talk Multithreading done right? by Rainer Grimm.

In most of the examples, two or more threads were writing to cout using the form:

cout << someData << "some string" << someObject << endl;

And one of the problems was that data sent from one thread often interrupted another thread, so the output was always messed up.
You can see the problem in this example:

#include <iostream>
#include <vector>
#include <thread>

using namespace std;

int main() {
    vector<thread> thr;
    for(int i = 0; i < 10; i++) {
        thr.emplace_back([i]() { cout << "thread " << i << endl; });
    }
    for(int i = 0; i < 10; i++) {
        thr[i].join();
    }
}

Which on a test execution produces things like this on CLang:

tththrhrereaeadad d  201

And this on Visual Studio:

thread 0 thread thread 2 1

I speculated with Marco Arena about possible solutions to the problem, and tried to find a simple and elegant solution in my way back.

I started designing a solution by giving myself some guidelines, here listed in order of importance:

1. Opt-in: The user doesn’t have to pay (no performance penalty if the feature is not used).
2. Readability: The user code must be self-explaining.
3. Simplicity of use: I wanted to provide the end-user with something that worked “out of the box” requiring as less code as possible.
4. Predictability: I wanted to maintain all the features and feature the standard streams have.
5. Concise solution: I didn’t want to write tons of code, I’m too lazy.

An intrusive solution (e.g. adding a mutex to each stream) would have violated the first principle, and quite definitely the fifth too.

Wrapping streams

The first solution I tried was a simple stream wrapper that locks the stream using RAII idiom:

class stream_locker {
    ostream& stream;
    lock_guard<mutex> guard;
public:
    stream_locker(ostream &s, mutex &m) : stream(s), guard(m) {}

    template<typename T> stream_locker& operator << (const T &x) {
        stream << x;
        return *this;
    }
};

Here’s the intended usage:

mutex m;
stream_locker(cout, m) << someData << "some string" << someObject << endl;

This didn’t work, because the type is not getting deduced for manipulators like endl (since it’s an overloaded function).

A solution to this problem could have been manually providing overloads for operator << for manipulators, such as:

class stream_locker {
    ...
    stream_locker& operator << (ostream& (*pf)(ostream&)) {
        stream << pf;
        return *this;
    }
 };

The big drawback with this solution was that we needed to provide an overload for every std manipulator type, like the ones declared in <ostream> and also any other manipulator contained in <iomanip>. Moreover, any overloaded operator << in other libraries or user code would have require additional effort (it violates both simplicity and predictability guidelines I gave myself). This was not getting anywhere.

Another possible solution was to return back using the ostream& as soon as possible, like after the first execution of operator <<.

class stream_locker {
    ostream& stream;
    lock_guard<mutex> guard;
public:
    stream_locker(ostream &s, mutex &m) : stream(s), guard(m) {}

    template<typename T> ostream& operator << (const T &x) {
        return stream << x;
    }
};

This was slightly better, but it would require, sometimes, to insert empty strings to achieve correct results. Consider the following:

stream_locker(cout, m) << value << " in octal is " << oct << value << dec << endl; // OK
stream_locker(cout, m) << oct << value << " in octal is " << dec << value << " in decimal." << endl; // ERROR
stream_locker(cout, m) << "" << oct << value << " in octal is " << dec << value << " in decimal." << endl; // OK

The second line won’t compile, since there’s no overload for operator << between a stream_locker and a manipulator like oct. The third will work, because inserting the empty string will result in calling our member operator <<, which now returns an ostream&.

We could definitely solve the problem by making the stream variable public, and changing the usage syntax slightly:

class stream_locker {
    lock_guard<mutex> guard;
public:
    ostream& stream;
    stream_locker(ostream &s, mutex &m) : stream(s), guard(m) {}
};

stream_locker(cout, m).stream << value << " in octal is " << oct << value << dec << endl; // OK
stream_locker(cout, m).stream << oct << value << " in octal is " << dec << value << " in decimal." << endl; // OK
stream_locker(cout, m).stream << "" << oct << value << " in octal is " << dec << value << " in decimal." << endl; // OK

This works, but the syntax looks horrible.

Lifetime of temporaries

Wait a second, why does this works? If we are return back to the ostream&, doesn’t our temporary stream_locker get destroyed right away, releasing the lock?

The answer is no. Any temporary variable created in a sub-expression have its lifetime extended to the “full-expression”, which in our case ends at the semicolon.

The stream_locker object is now just behaving exactly like a normal lock_guard<mutex>. Can’t we just use a lock_guard<mutex> directly?

cout << lock_guard<mutex>(m) << ... ;

To provide this syntax we just need to overload the operator <<:

inline ostream& operator<<(ostream& out, const lock_guard<mutex> &) {
    return out;
}

This solution doesn’t have the drawbacks of wrapper object, since no wrapper is created anywhere. The stream is returned right away, and the lock guard parameter is there for the only sake of including it in the streaming chain.

This definitely looks promising, but the syntax is still a bit too complex. Could we possibly simplify it?

I would definitely prefer something like:

cout << lock_with(m) << ... ;

Basically, lock_with should return a fresh new lock_guard locking our mutex m. Something like this:

template <typename T> lock_guard<T> lock_with(T &mutex) {
    return lock_guard<T>(mutex);
}

Unluckily, this doesn’t work, as you can see yourself by trying to compile the following snippet.

#include <mutex>
#include <iostream>

using namespace std;

inline ostream& operator<<(ostream& out, const lock_guard<mutex> &) {
    return out;
}

template <typename T> lock_guard<T> lock_with(T &mutex) {
    return lock_guard<T>(mutex);
}

int main() {
    mutex m;
    cout << lock_with(m) << "test" << endl;
}

Problem is, lock_guard is not copyable. Moving is also deleted, because this class implements the concept of “scoped ownership”. Moreover, the constructor taking a mutex is explicit, so we can’t exploit copy-list-initialization in our solution. Not with that constructor, at least.

As often happens, the solution to my problem was already on stack overflow, The link explains why lock_guard is not moveable, and it provides a nice workaround, using another constructor, taking two parameters (so we can use copy-list-initialization on that).

template <typename T> inline lock_guard<T> lock_with(T &mutex) {
    mutex.lock();
    return { mutex, adopt_lock };
}

The solution

Finally, we can test our original code with the final structure:

#include <mutex>
#include <vector>
#include <thread>
#include <iostream>

using namespace std;

inline ostream& operator<<(ostream& out, const lock_guard<mutex> &) {
    return out;
}

template <typename T> inline lock_guard<T> lock_with(T &mutex) {
    mutex.lock();
    return { mutex, adopt_lock };
}

int main() {
    mutex m;
    vector<thread> thr;
    for(int i = 0; i < 10; i++) {
        thr.emplace_back([i, &m]() { cout << lock_with(m) << "thread " << i << endl; });
    }
    for(int i = 0; i < 10; i++) {
        thr[i].join();
    }
}