Variadic template and tuples

Lets create a third logger, we call it ComposedLogger which can take arbitrary number of loggers, and for each debug call we will call each of the log instances with debug. Then we can use the composed Logger, with our complicated class.

By the above requirement we kind of recongnise that we need variadic templates. But not get ahead of ourselfs. Lets first checkout our Log classes that we are using.

 1: ///////////////////////////////////////////////////////////////////////////////
 2: //                                  PissLog                                  //
 3: ///////////////////////////////////////////////////////////////////////////////
 4: class PissLog
 5: {
 6: public:
 7:   PissLog() = default;
 8: 
 9:   template<class Args>
10:   void debug(std::string const& fmt, Args&& args) const
11:   {
12:     std::string newFmt = fmt::format("{:<20} {}\n", logName_, fmt);
13:     fmt::print(newFmt, std::forward<Args>(args));
14:   }
15: 
16:   template<class Args>
17:   void error(std::string const& fmt, Args&& args) const
18:   {
19:     std::string newFmt = fmt::format("{:<20} ERROR: {}\n", logName_, fmt);
20:     fmt::print(stderr, newFmt, std::forward<Args>(args));
21:   }
22: 
23:   std::string_view logName_{"PissLog"};
24: };
25: 
26: ///////////////////////////////////////////////////////////////////////////////
27: //                                   FmtLog                                  //
28: ///////////////////////////////////////////////////////////////////////////////
29: class FmtLog
30: {
31: public:
32: 
33:   template<class Args>
34:   void debug(const std::string& fmt, Args&& args) const
35:   {
36:     auto newFmt=fmt::format("{:<20} {}\n",logName_,fmt);
37:     fmt::print(newFmt,std::forward<Args>(args));
38:   }
39: 
40:   template<class Args>
41:   void error(std::string const& fmt, Args&& args) const
42:   {
43:     auto newFmt=fmt::format("{:<20} ERROR: {}\n",logName_,fmt);
44:     fmt::print(stderr,newFmt,std::forward<Args>(args));
45:   }
46: 
47:   std::string_view logName_{"FmtLog"};
48: };
49: 

As we can see there is no big difference from before, but instead of using NullLog we are using FmtLog, which is basically the same as PissLog, just for clarity.

Lets continue with the ComposedLogger we know it will use arbitrary number of loggers, and if we get a call to either debug or error method, then for each of the loggers-types that were provided (and instantiated) through the variadic template we need to call either debug or error, depending on the call.

 1: template<class... Logs>
 2: class ComposedLogger
 3: {
 4: public:
 5:   ComposedLogger()
 6:     : loggers_( Logs()... )
 7:   {}
 8: 
 9:   template<class Args>
10:   void debug(std::string const& fmt, Args&& arg)
11:   {
12:     log_debug(loggers_,
13:               fmt,
14:               std::forward<Args>(arg),
15:               std::index_sequence_for<Logs...>{});
16:   }
17: 
18:   template<class Args>
19:   void error(std::string const& fmt, Args&& arg)
20:   {
21:     log_error(loggers_,
22:               fmt,
23:               std::forward<Args>(arg),
24:               std::index_sequence_for<Logs...>{});
25:   }
26: 
27: private:
28:   std::tuple<Logs...> loggers_;
29: };
30: 

Line 1

The variadic template of Loggers. (e.g <PissLog,FmtLog> )

Line 6

This will instantiate each of the variadic templates types with the default constructor. std::tuple (1) has a direct constructor which initializes each element with corresponding parameters. Here is possible compiler rewrite using <FmtLog,PissLog> as parameters

1: ComposedLogger():
2:     loggers_{std::tuple<FmtLog,PissLog>(FmtLog(),Bepa())}
3: {}

The variadic template arguments are expanded and default initialized, and finally initialized into the loggers_ attribute.

Line 28

The tuple of Loggers, where each of the logger where instansiated on line 6.

Line 10

The debug method, this will call an additional function log_debug which we cover in the next section.

Line 15

Creates a index sequence which creates a compile-time sequence of indices. For our case we want to use the size of the tuple (or the number of items in our variadic template Logs), in other words create a index sequence from 0-sizeof(loggers). index_sequence_for is just an alias for template<class... T> using index_sequence_for = std::make_index_sequence<sizeof...(T)> Where T In our case is Logs. We need to provide a compile time sequence since its not possible to use runtime values when dealing with tuples. That is std::get<i>(tuple) in a for loop is not possible.

Line 19

And finally, basically the same as debug, though its calling error function instead.

Hopefully this was clear enough and that we straightend out the question marks regarding variadic templates. Lets now take a log at the Log_debug/error functions. These of course could be made as class method, but i choose to add them as stand-alone functions.

Lets dive right into it. This is what the debug function looks like. Actually this only works in c++14, since std::index_sequence is not valid in c++11. To have the same functionality in c++11 we need to use recursion (I think….).

1: template<class Tuple,class Args, std::size_t... Ls>
2: void log_debug(const Tuple& tpls,std::string const& fmt,Args&& arg, std::index_sequence<Ls...>)
3: {
4:    (void)(std::initializer_list<int>{
5:     ((void)std::get<Ls>(tpls).debug(fmt,std::forward<Args>(arg)),0)...
6:       });
7: }

Line 1

The template takes 3 parameters Tuple which is the tuple that consists of all the logs. Args which is the argument to the format function. And finally the variadic template argument Ls, which is of type std::size_t... where ... means it is a sequence of indices, e.g 0,1,2,3,4,5... , but even if we have a sequence we need to somehow use each of the sequence number to call std::get<Ls[N]>(tuple) to retrieve the tuple we want to use. (take a look at Variadic sequence section for some explanation),

Line 5

In this case we use std::initializer_list. The initializer_list (IL) is constructs a array type object of some type (int in this case). Since we're not actually intrested in the array type we discard the return value with (void). And since debug is not actually returning anything we need to provide a default return value (0). The good thing is that we use the std::index_sequence that we created to force a for_each behaviour.

For more information on this can be read here.

The std::initializer_list worked, but its kind of crude and hard to get right. Fortunatly c++17 came with another nice feature. The Fold expression, (see Variadic sequence on how it works), this feature made the std::initializer_list obsolete (for our circustance). So instead the function would look something like.

1: template<class Tuple,class Args, std::size_t... Ls>
2: void log_debug(const Tuple& tpls,std::string const& fmt,Args&& arg, std::index_sequence<Ls...>)
3: {
4:   (std::get<Ls>(tpls).debug(fmt,std::forward<Args>(arg)),...);
5: }

We still have the function definition with the template parameters, but we have removed the std::initializer_list with a fold expression. Much nicer and cleaner. The fold expression is only valid in c++17.

To stretch it even further, there is a cleaner way of doing this. This part we actually changed the definition of the function. Instead of using fold expression we can use std::apply, well actually we do use fold expression, but in another way. Since the index_sequence is no longer needed, we can grab the opportunity that the variadic template (size_t…) has been removed, and use it to define the types in our tuple instead. This way it gets even more safe and easier to understand. Another cool feature which is using parameter packs in auto for lambda. What this means is that we can use auto... to get arbitrary number of parameters. E.g

1: auto f = [](auto... s)
2:          {
3:            (fmt::print("{} ",s),...);
4:          };
5: f("Hello", "world", "Another day","in paradise",12,2.3,'\n');

Here we use the fold expression almost like a for_each with the arguments (s).

> main
Hello world Another day in paradise 12 2.3

This seems like a useful construct, but we need to use each tuple item as an argument to the function to be able to use it in our purpose. So it doesn't solve the problem completely, we need some other feature to send the tuple as arguments to the lambda function. Fortunatly c++17 provides us with just such a feature which is called std::apply. The idea behind std::apply is to invoke a callable object with a tuple of arguments. E.g

1: auto f = [](auto... s)
2:          {
3:            (fmt::print("{} ",s),...);
4:          };
5: auto tpl = std::make_tuple("Hello", "world", "Another day","in paradise",12,2.3,'\n');
6: std::apply(f,tpl);

This means we can use the tuple that we created as argument to our lambda. Neat! And really useful since now we can rewrite our log_debug in a really nice way. I devoted a small section for a more thorough example working with std::apply, lambdas and parameter packs.

 1: template<class Args,class... T>
 2: void log_debug( std::tuple<T...> tuple, std::string const& fmt, Args&& args)
 3: {
 4:   std::apply([fmt,args](auto const&... item)
 5:              {
 6: 
 7:                (item.debug(fmt,std::forward<Args>(args)),... );
 8:              },
 9:     tuple);
10: }

Since the definiton of the function is changed, we also need to change the call to the function, we don't need the index_sequence any more and the call would be much easier to understand.

 1: template<class... Logs>
 2: class ComposedLogger
 3: {
 4: public:
 5:   ComposedLogger()
 6:     : loggers_( Logs()... )
 7:   {}
 8: 
 9:   template<class Args>
10:   void debug(std::string const& fmt , Args&& arg)
11:   {
12:     log_debug(loggers_, fmt, std::forward<Args>(arg));
13:   }
14: 
15:   template<class Args>
16:   void error(std::string const& fmt , Args&& arg)
17:   {
18:     log_error(loggers_, fmt, std::forward<Args>(arg));
19:   }
20: private:
21:   std::tuple<Logs...> loggers_;
22: };

As described earlier this construct was really useful. But lets dive into some more elaborate constructs with this.

Here is a small snippet of a tuple-to-string

 1: template<class... T>
 2: std::string tuple_to_string(std::tuple<T...> const& tpl)
 3: {
 4: 
 5:   auto toString = [](auto val) -> std::string {
 6:     std::string strVal{ "Val:" };
 7:     if constexpr (std::is_arithmetic_v<decltype(val)>) {
 8:       return std::to_string(val) + "*";
 9:     } else {
10:       return std::string(val) + "#";
11:     }
12:   };
13: 
14:   std::string str;
15:   std::apply([&str, toString](auto... args)
16:              {
17:                str = (toString(args) + ...);
18:              }, tpl);
19: 
20:   return str;
21: }

Lets break it down.

Line 2

the function definition, recieves a tuple of types T... in other words its a tuple with arbitrary number of fields and types.

Line 5

A lambda function which converts to string. The argument is an auto which basically means it can be of any number of types, just like template<class T>. The function returns a string (->std::string).

Line 7

The if constexpr(expr) is a compile time construct, that means it will be evaluated during compile time. That also means that each of the types in the tuple needs to be known at compile time (which it is). The expression is evaluated using type traits and checks if its a arithmetic variable, that means its either floating type or integer type, in that case we need to convert it to a string (std::to_string). Any other sort is consider to be convertible to string via the std::string constructor.

Line 15

Invocation of std::apply, here we invoke a lambda by capturing

First of all lets take a look at a function which doesn't use sequence.

1: 
2: template<std::size_t t>
3: void f()
4: {
5:   fmt::print("t={}\n", t);
6: }

Not much to it, the std::size_t is provided as a compile-time constant to the function, and is then used with fmt::print. But what if I want to do the same thing, but with a sequence of numbers? We know that variadic templates can be used to have an arbitrary number of arguments, in fact it is also possible to send in an arbitrary number of a fundamental type. E.g

1: template<std::size_t... ts>
2: void fs()
3: {...}

Now to be able to use those values we need some way of doing for_each of ts. Thats where fold expressions comes in, these are c++17 constructs, and can be used with operator (+,-,&&,||,…..), they can be right folded or left folded.

Lets take an example of right folded.

 1: template<std::size_t... ts>
 2: void fs()
 3: {
 4:   auto p = (ts - ...);
 5: }
 6: 
 7: int main()
 8: {
 9:   fs<10,3,2>
10:   return 0;
11: }

This extracted to something like:

1: unsigned long p = 10UL - (3UL - (2UL - 1UL));
2: 10 - (3 - (1))
3: 7 + 1
4: 8

If we now switch this to left folded

 1: 
 2: template<std::size_t... ts>
 3: void fs()
 4: {
 5:   auto p = (ts - ...);
 6: }
 7: 
 8: int main()
 9: {
10:   fs<10,3,2>
11:   return 0;
12: }

Now the output would be something like

1: unsigned long p = ((10UL - 3UL) - 2UL) - 1UL;
2: (7 - 2 ) -1
3: 5 - 1
4: 4

The difference between Right and Left has a major impact when considering using some operators (e.g -). One possible solution is to use "," operator together with a function.

E.g

 1: void fun( std::size_t i)
 2: {
 3:   std::cout << i << '\n';
 4: }
 5: 
 6: template<std::size_t... ts>
 7: void fs()
 8: {
 9:   (...,fun(ts) );
10: }
11: 
12: int main()
13: {
14:  fs<10,3,2,1>();
15: }

The call would be as exaplained before

1: ./main
2: 10
3: 3
4: 2
5: 1
6: 
7: 
8: //Right folded ((fun(10UL) , fun(3UL)) , fun(2UL)) , fun(1UL) 
9: //Left folded  fun(10UL) , (fun(3UL) , (fun(2UL) , fun(1UL)));

In this case the Left folded and Right folded should end up with the same result.

user defined deduction

This feature can actually be extended a bit further. The problem with the above system, is that we cannot see which sensors/cameras that are failing. We could start iterating again, but that is a waste of time and performance. Instead we could use a capture to gain the list of failing sensors and cameras. But then we need somehow to send in a capture list of some sort. But then agin, by creating a new function using the argument that we can use as a capture and returning a lambda function should work. Lets do an example with failing sensors.

 1: namespace expressions {
 2: 
 3:   auto get_check_sensor(Security::Sensors& failedSensors)
 4:   {
 5: 
 6:     /////////////////////////////////////////////////////////////////////////////
 7:     //                      Check if sensors are turned on                     //
 8:     /////////////////////////////////////////////////////////////////////////////
 9: 
10:     return ([&failedSensors](auto security) {
11:       auto it = std::copy_if(std::begin(security.sensors_), std::end(security.sensors_),
12:                              std::begin(failedSensors),
13:                    [](auto const& sensor) {
14:                      return !sensor.verify();
15:                    });
16:       failedSensors.resize(std::distance(std::begin(failedSensors), it));
17:       return failedSensors.empty();
18: 
19:     });
20:   }
21: 
22: }  // expressions

This of course is one of doing, don't actually know if its the fastests. One thing not to forget is to make sure that failedSensors are large enough to fit all the sensors (if for some reason all sensors fails).

We can now use this togheter with our operators in the following way.

 1: Security::Sensors failedSensor(security.sensors_.size());
 2: auto all_system_on =  expressions::get_check_sensor(failedSensor) && expressions::check_camera_power;
 3: 
 4: if( !all_system_on(security) )
 5: {
 6:   for( const auto& sensor : failedSensor )
 7:   {
 8:     console->error(" Sensor \"{}\" failed  ", sensor.name_);
 9:   }
10: }

The exact same way can be done with cameras. This example shows that its possible to retrive output from the checks, its a side effect, but can be pretty useful. One thing to remeber is that failedSensor must outlive the call to the check-function.

Lets say I want to print out all the cameras which has are turned off. That is having power_ set to false.

Something like:

1: auto fun = expressions::get_all_cameras_prop(false) >>= expressions::print;
2: fun(security.cameras_.begin(), security.cameras_.end());

Lets first define the function which is neede to filter out, that is expressions::get_all_cameras_prop

1: auto get_all_item_prop( bool prop)
2: {
3:   return [prop](auto const& item) {return (item.power_ == prop ); };
4: }

This is hopefully self explanatory, all the cameras with the capture prop will return true, and therefor passed on to the next lambda.

So the next lambda will is the verb of the sentence. In this case we will use a print function.

1: auto print = [](auto const &item) {
2:     fmt::print("|{:<10}|{:>30}|\n","Item", item.name_);
3: };

The only things that is missing is to add the operator which is needed >>= (well, I don't know if anything else is better). Since we want to use this operator for other functions too, to iterate over ranges, we define it as taking in iterators instead.

 1: 
 2: namespace expressions {
 3: 
 4: template<class Expr1, class Expr2>
 5: auto operator>>=(Expr1&& lhs , Expr2&& rhs)
 6: {
 7:   return [Lhs = std::move(lhs), Rhs = std::move(rhs)](auto it_start,auto it_end)
 8:   {
 9:     std::for_each(it_start, it_end,
10:                   [&Lhs,&Rhs](auto const& item) {
11:                     if(Lhs(item))
12:                     {
13:                       Rhs(item);
14:                     }
15: 
16:                   });
17:   };
18: };
19: 
20: }  // expressions

In this case we can do the following:

 1: int main()
 2: {
 3:   Security security{
 4:     {Camera{Pos{1, 2}, false, "Camera 1"},
 5:      Camera{Pos{2, 3}, true, "Camera 2"},
 6:      Camera{Pos{3, 4}, false, "Camera 3"}
 7:     },
 8:     {Sensor{Pos{1, 2}, false, "Sensor 1"},
 9:      Sensor{Pos{2, 3}, true, "Sensor 2"},
10:      Sensor{
11:         Pos{3, 4}, false, "Sensor 3"
12:       }
13:     }
14:   };
15: 
16:   auto console = spdlog::stdout_color_mt("console");
17: 
18:   console->debug("Starting PassOn");
19: 
20:   {
21:     //auto f = expressions::get_all_cameras_prop(true) >>= expressions::get_print("Camera");
22:     fmt::print("|{:-^10}|{:-^30}|\n", "-",  "-");
23: 
24:     auto f = expressions::get_all_cameras_prop(true) >>= expressions::print;
25:     f(security.cameras_.begin(),security.cameras_.end());
26: 
27:     fmt::print("|{:-<10}|{:->30}|\n", "-",  "-");
28:     auto g = expressions::get_all_sensors_prop(false) >>= expressions::print;
29:     g(security.sensors_.begin(),security.sensors_.end());
30:     fmt::print("|{:-<10}|{:->30}|\n", "-",  "-");
31:   }
32: 
33: 
34:   return 0;
35: }

And when we run this program we get the following output

|----------|------------------------------|
|Item      |                      Camera 2|
|----------|------------------------------|
|Item      |                      Sensor 1|
|Item      |                      Sensor 3|
|----------|------------------------------|