Hans Ewetz: 2022

Ruthless in purpose insidious in method

-- Orphan Black TV Series, season 3 Episode 8

In this entry I will make a few comments on how the constraint machinery introduced in C++20 works. Most of the comments I have touches on issues which were not completely clear to me to when trying out the new C++20 constraint related constructs the first time. However, the content of this entry is fairly basic and should not introduce new ideas or clever solutions to problems.

I will not get into all the syntactic and semantic details of the C++20 constraint machinery. Instead I'll make an attempt to categorize various ideas so as to get a better understanding of how it all fits together.

Three types of basic constraint constructs

There are three(3) types of basic contraints used when constraining the interface of templates:

plain boolean expressions evaluated at compile time
concepts
requires expressions

Constraints with plain boolean expressions

Let's take a look at constraining a template parameter using a plain boolean expression:

template<typename T>
requires is_integral_v<T>
int foo(T t){
  return t+17;
}
foo(1);     // OK '1' is integral
foo(1.0);   // ERROR '1.0' is not integral

As expected we can overload foo if we want to support floating point variables:

template<typename T>
requires is_floating_point_v<T>
int foo(T t){
  return t+3.14;
}
foo(1.0);   // OK now

There is nothing new here as far as the boolean expression is concerned. Only the requires keyword is new which ties the requirement to the template.

Constraints with concepts

concept is a new keyword in C++20. A concept lets you assign a name to a collection of requirements. As mentioned earlier, a constraint comes as one of three types (or as a combination of them) and a concept can mix multiple constraints. For example:

template<typename T>
requires integral<T>
int foo(T t){
  return t+17;
}

Here integral is a concept which evaluates to either true or false. We can simplify the code slightly by constraining T directly in the template (replace typename with the name of the concept):

template<integral T>
int foo(T t){
  return t+17;
}

Constraints using requires expressions

The third type of constraint is a requires expression. A simple example is:

template<typename T>
requires requires(T t){
  t+1;
}
int foo(T t){
  return t+17;
}

Notice that requires is part of the requires expression. The first requires starts what is called a requires clause (see the following section). The second requires starts the require expression. There are four(4) types of conditions that can be expressed inside a requires expression:

simple requirement (the example above expressed a simple requirement)
type requirements
compound requirements
nested requirements

A simple requirement just checks that it states something that is correct from a language point of view. It does not evaluate the expression. An example is the initial example from this section:

template<typename T>
requires requires(T t){
  t+1;
}
int foo(T t){
  return t+17;
}

A type requirement starts off with the word typename followed by a type. It checks that the type is valid. For example:

template<typename C>
requires requires{
  typename C::value_type;
}
void foo(C&c){
  ...
}

A compound requirement allows us to specify certain properties of an expression. For example, we can specify that the result of dereferencing a variable is convertable to an int:

template<typename IT>
requires requires(IT it){
  {*it}->convertible_to<int>;
}
void foo(IT it){
  return *it;
}

There are two things to notice here. First, to the right of the arrow, we don't specify the type of the expression. Instead, we specify an expression evaluating to either true or false.

Second, the concept is_convertible takes two template parameters. However, in the context in which it is used, the first parameter is the type of result of the expression and cannot be explicitly specified in the parameter list to convertible_to.

This may look strange. However, it is no different than when we replace typedef with the name of a concept and do not explicitly specify the parameter to the concept:

template<integral T>
void foo(T t){
   ...
}

integral takes one parameter. But because of the context the parameter is T and we cannot explicitly specify it.

A nested requirement nests a requires expression inside another requires expression:

template<typename IT>
requires requires{
  requires same_as<IT,int*>;
}
...

Conjunctions and disjunctions

A constraints can be formed as the conjunctions of two constraints. For example:

template<typename T,typename U>
requires integral<U>&&integral<T>
void foo(T t,U u){
  ...
}

Here we require that both T and U are of integral type.

Disjunctions are similar with the exception that one one of the two participating constraints must be true. Disjunctions are formed using the || operator.

Requires clause

A requires clause is the clause that attaches constraints to template arguments or a function declaration. As shown in the example it starts with the keyword requires and is followed by a constraint.

Notice that both a requires clause and a requires expression start with a requires word. Therefore when introducing a requires expression as a constrint in a requires clause we end up with two requires words as:

template<typename T>
requires requires(T t){t+1;}
void foo(T t){
  return t+17;
}

constraints and auto

Constraints also work with auto declared parameters. An auto declared parameter is constrained by writing the requirement before the auto keyword.

As usual when comparing code using auto with template based code, we see that in the case of auto we have to write more code. For example, to get the type of a parameter we must often use decltype in conjunction with decay which makes the code more cryptic and less readable.

Let's now take a look and compare constraints on template parameters with constraints on code using auto declared parameter.

Using plain templates

Say we want to constrain a template function so that it is only called on collections supporting push_back of values. The syntax for a concept that supports this is fairly straight foward.

template<typename C>
concept SupportPushback=requires(C c){
  typename C::value_type;
  c.push_back(declval<typename C::value_type>());
};

We can use this concept in the implementation of a function template that adds values to a container:

// using plain templates
template<SupportPushback C,typename ...T>
void add(C&c,T const&...ts)
requires (convertible_to<T,typename C::value_type>,...)
{
  (c.push_back(ts),...);
}

For safety we added a bool expression that ensures that the values we insert are convertible to the value type of the container.

Using auto parameters

We can re-write this function template slightly, but this time with auto parameters:

// using auto
void add(SupportPushback auto&c,auto&&...ts)
requires (convertible_to<decay_t<decltype(ts)>,typename decay_t<decltype(c)>::value_type>,...)
{                                           // note: std::decay is mandatory so we can check convertibility                                                                                                             
  (c.push_back(std::forward<decltype(ts)>(ts)),...);                               
}

Clearly it becomes more verbose as well as slightly more messy.

Remember that decay converts from l-value to r-value after which it removes references as well as any top level cv-qualifications. It allows us to cleanly compare two types without cv-qualifications and references involved.

Notice that we constrain the auto. declared parameter by writing the name of the concept before auto Since we don't have a name for the type of the container nor the type of the variables we store in the container we have extract the type using decltype. To get the convertible_to to work we also have to use decay so that types can be compared properly or it will not compile correctly.

Constraints and `constexpr`

The C++ constraint constructs (concepts, requires expressions and obviously plain compile time boolean expressions) all evaluate to bool at compile time. This means that they can be used in constexpr conditional statements as an alternative to function overloading.

The following example shows how we can use concepts in conjunction with constexpr conditionals to map out the execution path during compilation:

template<typename C,typename T>
void add2collection(C&c,T const&val){
  // make sure the collection supports either push_back(...) or insert(...) of elements
  static_assert(SupportPushback<C>||SupportInsert<C>,"invalid collection type in 'add2collection'");
  
  // check if we have a push_back(...) or insert(...) collection
  if constexpr(SupportPushback<C>){
    c.push_back(val);
  }else
  if constexpr(SupportInsert<C>){
    c.insert(val);
  }
}

Constraints on non-templates

Yes, it is possible (and sometimes also meaningfull) to constrain functions that are non-templates. Here I'll show a mini demo example that at first might look like cheating since it is based on a class template. However, the functions are plain functions - not function templates:

...
template<typename T>
struct Kar{
  void foo()requires integral<T>{cout<<"integral"<<endl;};
  void foo()requires floating_point<T>{cout<<"floating_point"<<endl;};
};
...

  Kar<int>karInt;         // prints 'integral'
  Kar<double>karFloat;    // prints 'floating_point'
  karInt.foo();
  karFloat.foo();
  ...

Constraints and lambdas

Lambdas using template parameters as well lambdas using auto parameters can be constrained. However, I have difficulty in seing many use cases where this might be useful. Clearly I might be missing something here. In any case, I'll give a small example where we actually do overload resolution across a set of lambda functions.

Say we want to apply a visitor to a value in a variant object. The following idiomatic construct together with overloading using concepts lets us in a fairly simple but convoluted double dispatch way seek out the correct lambda function to call.

First we'll use an idiomatic construct defining a class containing multiple overloaded operator() functions:

...
// class holding multiple overloaded operator() functions
template<typename ... Ts> struct overloaded : Ts... { using Ts::operator()...; };
...

Next we'll create a set of lambda functions which we package inside a single function object:

   ...
  // callable function objects
  auto f_integral=[]<integral T>(T const&t){cout<<"integral"<<endl;};
  auto f_int=[](int t){cout<<"int"<<endl;};
  auto f_floating_point=[](floating_point auto const&t){cout<<"floating_point"<<endl;};
  auto f_string=[](string const&t){cout<<"string"<<endl;};
  auto f_push_back=[](SupportPushback auto const&t){cout<<"pushback"<<endl;};
  auto f_insert=[](SupportInsert auto const&t){cout<<"insert"<<endl;};

  // merge all callable objects into a single one having multiple operator()
  auto fvisit=overloaded{f_integral,f_int,f_floating_point,f_string,f_push_back,f_insert};
  ...

Finally we'll apply the function object using double dispatch to the the optional object:

  ...
  // for a variable 'val' visit the correct function object
  variant<long,int,double,string,vector<int>,set<string>>w=set<string>{};
  visit(fvisit,w);    // will call fvisit.operator()(SupportInsert auto const&t)
  ...

The output is:

insert

Ordering of constraints

The C++ reference https://en.cppreference.com/w/cpp/language/constraints defines how to impose an ordering of constraints:

A constraint P is said to subsume constraint Q if it can be proven that P implies Q up to the identity of atomic constraints in P and Q.

I won't get into the details here, but it suffices to say that the constraint P is narrower than the constraint Q if P subsumes Q. The (or one of the) benefit I can see of ordering of contraints is that if P subsumes Q, and P is satisfied, then P will be selected during overload resolution even though Q is also satisfied

Say we need a wrapper class storing a pointer to some data. We want to have the option of either storing the data as a raw pointer or as a smart pointer. In the destructor we want to delete the data if the pointer is a raw pointer. In the case of a smart pointer we want call release on the (smart) pointer. Depending on the behaviour of the smart pointer a release on the smart pointer may or may not delete the data held by the pointer.

Both raw and smart pointers act as pointers so we will impose a constrait that the pointer in the wrapper class satisfies a concept I named IsPointer. A smart pointer additionally has the feature of releaseing the held memory. Since the smart pointer is a pointer but has an additional constraint attached we say that the smart pointer constraint subsumes the raw pointer constraint and is therefore a more narrow constraint.

An example (that also incidentally is a better and more realistic example than my previous one of constraints applied to functions that are not function templates) will clarify.

Notice that in this implementation I've not made the effort that probably should be done to separate raw pointers from arrays. In real production code this should most likely be analysed and handled.

template<typename T>
concept IsPointer=requires{is_pointer_v<T>;};

template<typename T>
concept IsResettable=requires(T t){t.reset();};

template<IsPointer DATA>
class DataWrapper{
public:
  // ctor for moving/copying data pointer
  DataWrapper(DATA&&data):data_(std::forward<DATA>(data)){}
  DataWrapper(DATA const&data):data_(data){}

  // destructors for data satisfying 'pointer' requirement as well as for 'pointer AND resettable' requirement
  ~DataWrapper()requires IsPointer<DATA>{delete data_;}
  ~DataWrapper()requires IsPointer<DATA> && IsResettable<DATA>{data_.reset();}
private:
  DATA data_;
};
...
  int*rp=new int(100);                  // store a pointer to an int and delete memory when (first) destructor is called
  DataWrapper fd1(rp);
  auto sp=make_shared<int>(200);        // store a shared_ptr<> holding an int pointer and call reset on shared_pointer<> when (second) destructor is called
  DataWrapper d2(sp);
  auto up=make_unique<int>(300);        // store a unique_ptr<> to an int and call reset on pointer when unique_ptr<> when (second) destructor is called
  DataWrapper d3(std::move(up));
...

The destructors is where the interesting thing happends. The constraint in the second destructor subsumes the constraint in the first destructor and is chosen by the compiler for smart pointers.

Finally, it is important to undesratnd that for a constraint to subsume another constraint, the constraints must be expressed as concepts. The constraints cannot be expressed as requires expressions or plain boolean expressions.

Benefits of using constraints

Over time, I'm positve that the new constraint mechinery in C++20 will flush out a number of design paradigms as well as idiomatic constructs combining various C++ features that are not obvious at this point in time. In general, right now with limited knowldege, I would say that the biggest benefit of concepts and contraints is overload resolution. Overload resolution with constraints gives us clear un-convoluted scalpel like precision when controlling what functions to call without resorting to indirect constructs that happends to constrain template parameters by convoluted side effects.

C++20 concepts are sometimes touted as a panacea for better error messages. I do agree with that better error messages are sometimes a benefit of concepts and constraints - but not always. Make eneough errors in template based code and you will still generate possibly hundreds of lines, or thousands of lines of error messages.

A soft benefit of concepts and constraints is that it gives us a semi formal way of making the intent of template parameters more explicit. The trick here is to not go down the path where we focus on the syntax that is allowed, but more on the semantics of template parameters. Clearly, it is not possible to in a formal way specify the semantics of template parameters, but via the syntax we can make an attempt.

For example, in a previous section where we specified that a collection is either a SupportPushback or a SupportInsert colection, we specified the constraints using syntax. However, the intent is to separate between the semantics of adding a (possibly duplicate) value at the end of a collection and adding a unique value to the collection. Possibly for this semantic we should have used different names for the concepts. However, in C++ the semantics of push_back and insert on collections are fairly well agreed on.

A mini library for easy streaming of various collections

When printing debug messages, it's often convenient to print elements of a collections inline in a cout statement. Having files littered with ad-hoc code printing out elements in collections quickly becomes messy.

A library with a single function strcat(...) could be used as follows where the first parameter is the separator to be used between elements in the collection being printed:

  ...
  tuple<int,string,double>mytuple(1,"Hello",3.14);
  optional<int>myoptional(17);
  set<string>myset{"1","2","3"};
  cout<<"my tupple: ["<<strcat(", ",mytuple)<<"], myoptional: ["<<strcat(", ",myoptional)<<"], myset: ["<<strcat(", ",myset)<<"]"<<endl;
  ...

The output becomes:

my tupple: [1, Hello, 3.14], myoptional: [17], myset: [1, 2, 3]

There are of course many ways this can be implemented. Combining overloaded functions templates with the C++20 constraint machinery turns out to be suprisingly straight forward.

We'll start with a support concept that checks if a list of types can be streamed:

// check if types in a type pack are all streamable
template <typename...Ts>
concept Streamable=requires(std::ostream&os,Ts...ts){
  {((os<<ts),...)}->std::same_as<std::ostream&>;
};

Next, we'll implement a support function making std::pair streamable:

// declaration: make 'pair' streamable
template<Streamable S1,Streamable S2>
std::ostream&operator<<(std::ostream&os,std::pair<S1,S2>const&p);

We'll treat the class std::optional as a collection having zero or one elements:

// merge a single (or no)_ value from an optional parameter into a string
// (note: the 'sep' parameter is never used since an 'optional' (collection) has at most one element)
template<Streamable S,Streamable T>
[[nodiscard]]std::string strcat(S const&sep,std::optional<T>const&p){
  std::stringstream ret;
  if(p)ret<<p.value();
  return ret.str();
}

In the implementation below, the call to boost::lexical_cast should be updated to use 'normal' streaming to a std::stringstream for consitency (I have simply not done this due to time constraints). Additionally, the implemenation should probably be extended to support a few more data structures.

The remainder of the implementation follows here:

// merge streamable values from a collection with separator in between elements
// (we don't want to use this function if C<T> is a string, instead we want the 'strcat(S, T, Ts ...)
template<Streamable S,Streamable T,template<class>class C>
requires requires(S s,C<T>c,std::ostream&os){
  begin(c);
  os<<s;
  os<<*begin(c);
  begin(c)!=end(c);
}
[[nodiscard]]std::string strcat(S const&sep,C<T>const&c){
  std::stringstream ret;
  for(auto it=begin(c);it!=end(c);){
    ret<<*it;
    if(++it!=end(c))ret<<sep;
  }
  return ret.str();
}
// merge streamable values from a pack into a string with separator between values
template<Streamable S,Streamable T,Streamable...Ts>
[[nodiscard]]std::string strcat(S const&sep,T const&t,Ts const&...ts){
  std::stringstream ret;
  ret<<t,((ret<<sep<<ts),...);
  return ret.str();
}
// merge streamable tuple values into a string with separator between values
template<Streamable S,Streamable...Ts>
[[nodiscard]]std::string strcat(S const&sep,std::tuple<Ts...>const&tu){
  constexpr std::size_t LEN=sizeof...(Ts);               // get length of tuple
  std::stringstream ret;                                 // write tuple to a string stream
  std::string ssep=boost::lexical_cast<std::string>(sep);// convert sep to string
  std::string emptysep;                                  // get empty separator as a string
  [&]<std::size_t...Is>(std::index_sequence<Is...>){     // lambda template writing tuple elements
    ((ret<<std::get<Is>(tu)<<(Is<(LEN-1)?ssep:emptysep)),...); // don't write separator after last element
  }(std::make_index_sequence<LEN>());                    // call lambda and pass an index sequence as parameter so we can walk through tuple elements
  return ret.str();                                      // get string from string stream
}
// merge streamable pair value into a string with separator between values
template<Streamable S,Streamable T1,Streamable T2>
[[nodiscard]]std::string strcat(S const&sep,std::pair<T1,T2>const&p){
  std::stringstream ret;                                 // write tuple to a string stream
  std::string ssep=boost::lexical_cast<std::string>(sep);// convert sep to string
  ret<<p.first<<ssep<<p.second;
  return ret.str();                                      // get string from string stream
}

Hans Ewetz

Wednesday, June 1, 2022

A few notes on the C++20 constraints machinery

Three types of basic constraint constructs

Constraints with plain boolean expressions

Constraints with concepts

Constraints using requires expressions

Conjunctions and disjunctions

Requires clause

constraints and auto

Using plain templates

Using auto parameters

Constraints and `constexpr`

Constraints on non-templates

Constraints and lambdas

Ordering of constraints

Benefits of using constraints

A mini library for easy streaming of various collections

Wednesday, June 1, 2022

A few notes on the C++20 constraints machinery

Three types of basic constraint constructs

Constraints with plain boolean expressions

Constraints with concepts

Constraints using requires expressions

Conjunctions and disjunctions

Requires clause

constraints and auto

Using plain templates

Using auto parameters

Constraints and constexpr

Constraints on non-templates

Constraints and lambdas

Ordering of constraints

Benefits of using constraints

A mini library for easy streaming of various collections

Constraints and `constexpr`