Weird behavior of range-based for

camper

SAn schrieb:

So, what should I do?
We here use GCC 4.6.3.

Don't use ~~rvalue-ranges~~ initializer-lists directly.

i.e.

auto range = {My{}, My{}, My{}};
std::cout << "starting for loop" << std::endl;
for(My const &i: range) {
    std::cout << i.x << std::endl;
}

krümelkacker

Interesting. To be honest, it does not surprize me that different compiler (versions) handle this differently. I'm not sure whether I fully grasp the magic of initialiter_list with respect to life-time by now or not. On one hand a good compiler is supposed to be smart enough to put constant expression values into a read-only memory in cases like these:

vector<int> foo = {2,3,5,7,11};

the array containing the values would have a static life-time and reside in some read-only memory block.

In other cases we have non-literal types and dynamic initialization (like in your case) where a temporary array is created on the stack.

In any case, the life-time of the array is required to be at least as long as the life-time of the std::initializer object -- as far as I understand. I guess that copying the initializer_list object does not count and might produce a "dangling initializer_list" -- for example, if you return such a thing from a function. But then, what about RVO?

In the case of the for-range loop we have the following situation:

auto&& range = {My(), My(), My()};

because the for-range loop is syntactic sugar which when expanded contains this kind of line of code. Deduction rules say that the declared type of range will be initializer_list<My>&& . Then, I would expect the life-time extension rule for temporary objects (the initializer_list object) to kick in. And since the life-time of the initializer_list object is extended to match the scope of range , the array this object refers to should also exist in this scope (and possibly beyond depending on whether it is possible to put it into a static read only block).

So, I guess your old G++ version gets it wrong.

Here's a workaround:

My range[] = {My(), My(), My()};
for (My const& i : range) {
  cout << i.x
}

(untested)

krümelkacker

Come to think of it, your initializer list does not depend on any function parameters. But your element type has user-defined constructors and destructors. Does this mean that a compiler is allowed to implement

vector<My> foo()
{
  return {My(),My()};
}

like this

vector<My> foo()
{
  static My hidden_array[] = {My(),My()}; // static storage!
  return internal_array_to_ilist_detail(hidden_array);
}

or does it have to use an array on the stack such as

vector<My> foo()
{
  My hidden_array[] = {My(),My()}; // automatic storage!
  return internal_array_to_ilist_detail(hidden_array);
}

?

The difference is that in the first case, the initialization is only done once and destructors are called when the program ends.

Someone please explain the semantics of initializer_lists with respect to the elements life-time to me. Thanks.

camper

n3337 schrieb:

8.5.4 / 6
The lifetime of the array is the same as that of the initializer_list object.

g++ apparently doesn't consider 12.2/5 (and possibly 12.2/4) in this context so gets this wrong. That would be a bug (this part was in the draft at least from n2914 on - june 2009).

Sone

Lets have a look at the "equivalence" to every range-based for loop from the standard N3337 itself:

{
    auto && __range = range-init; // in our case, auto deduces to std::initializer_list (type of __range is std::initializer_list<...>&&)
    for ( auto __begin = begin-expr, // begin-expr is std::begin(__range)
               __end = end-expr;  // dito, std::end
         __begin != __end;
         ++__begin ) 
    {
        for-range-declaration = *__begin; // asignment
        statement // unimportant (can be compund-statement o.s.)
    }
}

Applied to our example:

#include <iostream>

struct My
{
     My() { std::cout << "Constructed: " << this << '\n'; }
    ~My() { std::cout << "Destructed: "  << this << '\n'; }
};

int main()
{
    auto && __range = {My(), My(), My()};
    for ( auto __begin = std::begin(__range),
               __end = std::end(__range);
         __begin != __end;
         ++__begin )
    {
        std::cout << "statement!\n";
    }
}

Now, what does this code print on GCC 4.8:

Constructed: 0x22fe20
Constructed: 0x22fe21
Constructed: 0x22fe22
statement!
statement!
statement!
Destructed: 0x22fe22
Destructed: 0x22fe21
Destructed: 0x22fe20

GCC 4.7.2 (from Ideone):

Constructed: 0xbfef3b9d
Constructed: 0xbfef3b9e
Constructed: 0xbfef3b9f
statement!
statement!
statement!
Destructed: 0xbfef3b9f
Destructed: 0xbfef3b9e
Destructed: 0xbfef3b9d

GCC 4.6.2:

Constructed: 0x28ff0d
Constructed: 0x28ff0e
Constructed: 0x28ff0f
Destructed: 0x28ff0f
Destructed: 0x28ff0e
Destructed: 0x28ff0d
statement!
statement!
statement!

What i don't understand is, how this is even possible. The init-list is destroyed, before i even iterate through it? Or could this be an optimization - but the question is, how could this be, why would newer compilers NOT optimize this... :head-scratch:

P.S.: SAn, do you use a 32-Bit Compiler on a 64-Bit machine?

Nexus

Sone, what happens if you also log copy/move constructors and assignment operators?

Sone

Nexus schrieb:

Sone, what happens if you also log copy/move constructors and assignment operators?

Same shit.

#include <iostream>

struct My
{
     My() { std::cout << "Constructed " << this << '\n'; }
    ~My() { std::cout << "Destructed "  << this << '\n'; }
    My(My const& a) { std::cout << "Copied "  << &a << " to " << this << '\n'; }
    My(My&& a) { std::cout << "Moved "  << &a << " to " << this << '\n'; }
    My& operator=(My&& a) { std::cout << "Move-assigned "  << &a << " to " << this << '\n'; return *this; }
    My& operator=(My const& a) { std::cout << "Assigned "  << &a << " to " << this << '\n'; return *this; }
};

int main()
{
    auto && __range = {My(), My(), My()};
    for ( auto __begin = std::begin(__range),
               __end = std::end(__range);
         __begin != __end;
         ++__begin )
    {
        std::cout << "statement!\n";
    }
}

camper

It should be rather obvious what happens:

auto && __range = {My(), My(), My()};

This creates 2 objects
- a temporary initializer_list object
- an array the initializer_list refers to
If the initializer_list object was not bound to a reference as is the case here it would be destroyed at the end of the initialization (=full expression, 12.2/3).
Because the initializer_list is actually bound to a reference its lifetime is extended to the lifetime of the reference, i.e. its scope.
The array refered to by the initializer_list object is required to have the same lifetime, but g++ 4.6.3 does not do this correctly.

auto range = {My(), My(), My()};

works, because here the lifetime of the initializer_list is automatic in the first place.
Despite the syntax this does not involve an elided copy of a initializer_list object, and if it did, the lifetime of the array would be too short.

SAn

The Clang has similar problem too.

You say about some array. I thought the initialiser list

auto a = {4, 3, 2, 1};

is equal to something like that:

std::initializer_list<int, 4> const a(4, 3, 2, 1);

Now I suspect the following:

int const t[] = {4, 3, 2, 1};
std::initializer_list<int> const a(t, t + 4);

Am I right?

The bug was discovered by our female coworker. She wrote something like that:

void process(cv::Mat &image1, cv::Mat &image2, cv::Mat &image3)
{ //cv::Mat is OpenCV image. On object copy it copies only header (image data is reference counted).
  for ( cv::Mat &image: {image1, image2, image3} )
  {
    ...do some common preprocessing...
  }

  ...do specific processing for image1...
  ...do specific processing for image2...
  ...do specific processing for image3... 
}

Sone

Am I right?

Nope. The Array is internal. (Btw, initializer_list has no such ctor)

std::initializer_list<int, 4> const a(4, 3, 2, 1);

What you mean is std::array (and it uses aggregate initialization, which requires curled braces)

Ich habe mich zwar bisher zurückgehalten, aber jetzt ist das Fass übergelaufen.

Ist es wirklich nötig, dass deutsche User in einem deutschen Forum englisch Reden?

Es gibt genug englische Programmier-Foren, und, ich sage das jetzt ungern, die sind z. T. noch besser als dieses hier.

Ach ja: und noch besser ist es natürlich, wenn man einen Thread mit englischem Titel erstellt, und in deutsch diskutiert. So stoßen Englischspracheige über google auf diesen Thread und verstehen nichts. Aber das nur nebenbei.

camper

SAn schrieb:

Now I suspect the following:
int const t[] = {4, 3, 2, 1};
std::initializer_list<int> const a(t, t + 4);
Am I right?

in essence (pseudo-code). Of course, if the initializer_list ist created implicitely it is temporary as is the the array it referes to. It's the compiler's job to keep them in sync.

Copying an initializer_list does not copy the array it uses nor does it magically extend the lifetime of said array. Doing so in a situation where the copy might outlive the original (for example: storing one in a container) is a bad idea.

Sone

Copying an initializer_list does not copy the array it uses nor does it magically extend the lifetime of said array.

WTF?
That can't be true. Whait a sec.

http://ideone.com/MMSJNJ
http://ideone.com/RGvAP1
So what are you talking about?

camper

Sone schrieb:

Copying an initializer_list does not copy the array it uses nor does it magically extend the lifetime of said array.

WTF?
That can't be true. Whait a sec.

http://ideone.com/MMSJNJ
http://ideone.com/RGvAP1
So what are you talking about?

what did you prove?
consider http://ideone.com/ftEcyd

Sone

camper schrieb:

Sone schrieb:

Copying an initializer_list does not copy the array it uses nor does it magically extend the lifetime of said array.

WTF?
That can't be true. Whait a sec.

http://ideone.com/MMSJNJ
http://ideone.com/RGvAP1
So what are you talking about?

what did you prove?

That the array the initializer_list is using, is indeed copied. Which is the exact opposite of what you said.

But apparently i didn't prove anything?

camper

Sone schrieb:

But apparently i didn't prove anything?

I trust you do know what undefined behaviour is.
For example, to pick over the dead bones.

Sone

Oh wait! The stack is not overriden, thus the values stay the same? ... or what ...

Sone

N3337 18.9 / 1

Copying an initializer list does
not copy the underlying elements.

g'dammit

camper

Sone schrieb:

N3337 18.9 / 1

Copying an initializer list does
not copy the underlying elements.

g'dammit

That's only a note though. You might want to read 8.5.4/5 et seq. to figure out how to derive that answer from the normative text (hint: consider what's not written in there).