<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[cvs parser really slow]]></title><description><![CDATA[<p>Hi, i m tring to parse a csv file into a Matrix.</p>
<p>My code works fine but is really slow since the file has about 12000 rows and 26 Columns<br />
is the anithing i can do to get this work faster mny thnks</p>
<p>vector&lt;vector&lt;string&gt;&gt; GetFileToMatrix(){<br />
ifstream file;<br />
vector&lt;vector&lt;string&gt;&gt; output;</p>
<p>string line;<br />
int counter =0;</p>
<p>file.open(this-&gt;filename_);<br />
while(!file.eof())<br />
{<br />
getline(file,line);<br />
istringstream is;<br />
is.str(line);<br />
output.resize(counter+1);<br />
while (getline(is,line,this-&gt;delimiter_)){</p>
<p>output[counter].push_back(line);<br />
}</p>
<p>counter++;<br />
}<br />
file.close();<br />
return output;<br />
}</p>
]]></description><link>https://www.c-plusplus.net/forum/topic/171825/cvs-parser-really-slow</link><generator>RSS for Node</generator><lastBuildDate>Thu, 17 Sep 2026 14:10:37 GMT</lastBuildDate><atom:link href="https://www.c-plusplus.net/forum/topic/171825.rss" rel="self" type="application/rss+xml"/><pubDate>Mon, 29 Jan 2007 18:53:30 GMT</pubDate><ttl>60</ttl><item><title><![CDATA[Reply to cvs parser really slow on Mon, 29 Jan 2007 18:53:30 GMT]]></title><description><![CDATA[<p>Hi, i m tring to parse a csv file into a Matrix.</p>
<p>My code works fine but is really slow since the file has about 12000 rows and 26 Columns<br />
is the anithing i can do to get this work faster mny thnks</p>
<p>vector&lt;vector&lt;string&gt;&gt; GetFileToMatrix(){<br />
ifstream file;<br />
vector&lt;vector&lt;string&gt;&gt; output;</p>
<p>string line;<br />
int counter =0;</p>
<p>file.open(this-&gt;filename_);<br />
while(!file.eof())<br />
{<br />
getline(file,line);<br />
istringstream is;<br />
is.str(line);<br />
output.resize(counter+1);<br />
while (getline(is,line,this-&gt;delimiter_)){</p>
<p>output[counter].push_back(line);<br />
}</p>
<p>counter++;<br />
}<br />
file.close();<br />
return output;<br />
}</p>
]]></description><link>https://www.c-plusplus.net/forum/post/1219354</link><guid isPermaLink="true">https://www.c-plusplus.net/forum/post/1219354</guid><dc:creator><![CDATA[alkatal]]></dc:creator><pubDate>Mon, 29 Jan 2007 18:53:30 GMT</pubDate></item><item><title><![CDATA[Reply to cvs parser really slow on Mon, 29 Jan 2007 19:21:12 GMT]]></title><description><![CDATA[<ol>
<li>
<p>vector&lt;vector&lt;string&gt; &gt; is really slow - if you use it you'll have to live with it. if you want something faster i'd say it depends on your data what would be optimal.</p>
</li>
<li>
<p>use a hand-written parser to go through the input file byte by byte.</p>
</li>
</ol>
<p>----</p>
<p>some very simple changes you might try, assuming all the lines in the input have the same length (it'll work even if they have not, but waste some memory):</p>
<pre><code class="language-cpp">vector&lt;vector&lt;string&gt; &gt; GetFileToMatrix()
{
    ifstream file; 
    vector&lt;vector&lt;string&gt; &gt; output; 

    string line; 
    int guess_elements = 0;
    int elements = 0;

    file.open(this-&gt;filename_); 

    while(!file.eof()) 
    { 
        getline(file,line); 
        istringstream is; 
        is.str(line); 

        //output.resize(counter+1);  // resize will probably never grow the vector beyond &quot;counter+1&quot; - so push_back should be a lot faster
        output.push_back(vector&lt;string&gt;());

        // preallocate space for the elements
        if(guess_elements &gt; 0)
            output.back().reserve(guess_elements);

        elements = 0;
        while (getline(is,line,this-&gt;delimiter_)){ 
            output.back().push_back(line); 
        } 
        if(elements &gt; 0)
            guess_elements = elements;
    } 
    file.close(); 
    return output; 
}
</code></pre>
<p>You might also try to move the istringstream out of the loop, to avoid construction/destruction for eache line. Especially if the implementation only reallocates the string buffer if it's too small that would save a lot of time.</p>
]]></description><link>https://www.c-plusplus.net/forum/post/1219370</link><guid isPermaLink="true">https://www.c-plusplus.net/forum/post/1219370</guid><dc:creator><![CDATA[hustbaer]]></dc:creator><pubDate>Mon, 29 Jan 2007 19:21:12 GMT</pubDate></item><item><title><![CDATA[Reply to cvs parser really slow on Mon, 29 Jan 2007 19:50:00 GMT]]></title><description><![CDATA[<p>vector&lt;vector&lt;string*&gt;*&gt; should be much better and when the methode returns the vector, the whole data is copied, not good for the performance.</p>
<p>And i dont think that streams are good for the perfomance. Reading the File raw would be much better.</p>
]]></description><link>https://www.c-plusplus.net/forum/post/1219389</link><guid isPermaLink="true">https://www.c-plusplus.net/forum/post/1219389</guid><dc:creator><![CDATA[Lars]]></dc:creator><pubDate>Mon, 29 Jan 2007 19:50:00 GMT</pubDate></item><item><title><![CDATA[Reply to cvs parser really slow on Mon, 29 Jan 2007 20:14:31 GMT]]></title><description><![CDATA[<p>hi mny thnks<br />
what datastructure would be optimat :<br />
the data is quite simple :</p>
<p>379,w,w,w,w,w,w,g,w,w,w,w,w,w,w,,w,......</p>
<p>something like that</p>
]]></description><link>https://www.c-plusplus.net/forum/post/1219407</link><guid isPermaLink="true">https://www.c-plusplus.net/forum/post/1219407</guid><dc:creator><![CDATA[alkatal]]></dc:creator><pubDate>Mon, 29 Jan 2007 20:14:31 GMT</pubDate></item><item><title><![CDATA[Reply to cvs parser really slow on Mon, 29 Jan 2007 20:48:50 GMT]]></title><description><![CDATA[<p>mny thnks hustbaer,<br />
the isstream out of the loop made it worlk well about 5 sec on a centrino with 1gb Ram.<br />
i couldn t compile the pointer version vector&lt;vector&lt;string*&gt;*&gt;.<br />
since the puschback didn t like the string.<br />
anyway mny thnks</p>
]]></description><link>https://www.c-plusplus.net/forum/post/1219438</link><guid isPermaLink="true">https://www.c-plusplus.net/forum/post/1219438</guid><dc:creator><![CDATA[alkatal]]></dc:creator><pubDate>Mon, 29 Jan 2007 20:48:50 GMT</pubDate></item><item><title><![CDATA[Reply to cvs parser really slow on Tue, 30 Jan 2007 08:58:48 GMT]]></title><description><![CDATA[<p>hi , sry for the 2 threads<br />
if i move istringstream is out of the loop i only get the first Column</p>
<p>any help</p>
]]></description><link>https://www.c-plusplus.net/forum/post/1219593</link><guid isPermaLink="true">https://www.c-plusplus.net/forum/post/1219593</guid><dc:creator><![CDATA[alkatal]]></dc:creator><pubDate>Tue, 30 Jan 2007 08:58:48 GMT</pubDate></item></channel></rss>