cvs parser really slow

Hi, i m tring to parse a csv file into a Matrix.

My code works fine but is really slow since the file has about 12000 rows and 26 Columns
is the anithing i can do to get this work faster mny thnks

vector<vector<string>> GetFileToMatrix(){
ifstream file;
vector<vector<string>> output;

string line;
int counter =0;

file.open(this->filename_);
while(!file.eof())
{
getline(file,line);
istringstream is;
is.str(line);
output.resize(counter+1);
while (getline(is,line,this->delimiter_)){

output[counter].push_back(line);
}

counter++;
}
file.close();
return output;
}

hustbaer

vector<vector<string> > is really slow - if you use it you'll have to live with it. if you want something faster i'd say it depends on your data what would be optimal.
use a hand-written parser to go through the input file byte by byte.

----

some very simple changes you might try, assuming all the lines in the input have the same length (it'll work even if they have not, but waste some memory):

vector<vector<string> > GetFileToMatrix()
{
    ifstream file; 
    vector<vector<string> > output; 

    string line; 
    int guess_elements = 0;
    int elements = 0;

    file.open(this->filename_); 

    while(!file.eof()) 
    { 
        getline(file,line); 
        istringstream is; 
        is.str(line); 

        //output.resize(counter+1);  // resize will probably never grow the vector beyond "counter+1" - so push_back should be a lot faster
        output.push_back(vector<string>());

        // preallocate space for the elements
        if(guess_elements > 0)
            output.back().reserve(guess_elements);

        elements = 0;
        while (getline(is,line,this->delimiter_)){ 
            output.back().push_back(line); 
        } 
        if(elements > 0)
            guess_elements = elements;
    } 
    file.close(); 
    return output; 
}

You might also try to move the istringstream out of the loop, to avoid construction/destruction for eache line. Especially if the implementation only reallocates the string buffer if it's too small that would save a lot of time.

Lars

vector<vector<string*>*> should be much better and when the methode returns the vector, the whole data is copied, not good for the performance.

And i dont think that streams are good for the perfomance. Reading the File raw would be much better.

hi mny thnks
what datastructure would be optimat :
the data is quite simple :

379,w,w,w,w,w,w,g,w,w,w,w,w,w,w,,w,......

something like that

mny thnks hustbaer,
the isstream out of the loop made it worlk well about 5 sec on a centrino with 1gb Ram.
i couldn t compile the pointer version vector<vector<string*>*>.
since the puschback didn t like the string.
anyway mny thnks

hi , sry for the 2 threads
if i move istringstream is out of the loop i only get the first Column

any help