FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

MySQL 5.1 and D DBI Row problem

 
Post new topic   Reply to topic     Forum Index -> DDBI
View previous topic :: View next topic  
Author Message
lars_kirchhoff



Joined: 27 May 2008
Posts: 9
PostPosted: Tue May 27, 2008 5:39 am    Post subject: MySQL 5.1 and D DBI Row problem Reply with quote
Hello everybody,

I'm very new to D, but I managed to setup everything so that I'm able to compile simple D sources and connect to a mysql database. I can do basic queries. But although I can access the row array with the results and output the field values by using a numerical index, I have problems to use the fieldname as key in the row array. Only the first two fieldnames are ok, but all others are missing respectively have different values then the names of the fields.

I did an output of the fieldnames in the MysqlResult.d file from the DDBI trunk (Line 49) (I got the latest snapshot from the svn):

Code:
Stdout("Keynames: ")(fields[index].name).newline;


It seems that only the first two fieldnames are set correctly and the rest of the fieldnames are translated to the name of the table and the name of the database and then zero..

Code:

field1 = fieldname 
field2 = fieldname
field3 = tablename
field4 = tablename
field5 = databasename
field6 = def
field7... =


I am using libmysqlclient.so.15.0.0 with the following build options:

Code:

dsss build -version=MySQL_51 -version=dbi_mysql -L/usr/local/mysql/lib/mysql/libmysqlclient.so.15.0.0


Is this a bug in the libmysqlclient version or is there a bug in the DDBI code? Any hint would be great...

best
Lars

Last edited by lars_kirchhoff on Tue May 27, 2008 9:39 am; edited 1 time in total
Back to top
View user's profile Send private message
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway
PostPosted: Tue May 27, 2008 6:44 am    Post subject: Reply with quote
I wouldn't be surprised if this is a bug in DDBI as it has been through some changes without too much testing. It is further likely that Row as it is today will change more in the near future, see thread on that in these forums - although it is slightly out of date.
Back to top
View user's profile Send private message
lars_kirchhoff



Joined: 27 May 2008
Posts: 9
PostPosted: Wed May 28, 2008 9:25 am    Post subject: Reply with quote
Thanks for the quick reply. I am not sure if it is a bug of the DDBI code as the wrong field names are already in the fields variable right after the mysql fetch fields function is used in line 22 of MysqlResult.d. This would actually indicate that the mysql_fetch_fields function returns something wrong. Therefore I tried different mysqlclient libraries, but unfortunately with the same result.

Code:
fields = mysql_fetch_fields(results);


So if it would be a DDBI bug, this would mean that something is wrong with the results variable, but I'm to new to D to see any bug there.

/lars
Back to top
View user's profile Send private message
lars_kirchhoff



Joined: 27 May 2008
Posts: 9
PostPosted: Tue Jul 22, 2008 5:47 am    Post subject: Reply with quote
Anything new here?
Back to top
View user's profile Send private message
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway
PostPosted: Tue Jul 22, 2008 6:59 am    Post subject: Reply with quote
I am progressing in the work, but it has been very slow over the summer - sorry about that.

Will get back to you.
Back to top
View user's profile Send private message
lars_kirchhoff



Joined: 27 May 2008
Posts: 9
PostPosted: Tue Jul 22, 2008 8:17 am    Post subject: Reply with quote
thanks...
Back to top
View user's profile Send private message
lars_kirchhoff



Joined: 27 May 2008
Posts: 9
PostPosted: Thu Jul 24, 2008 5:05 am    Post subject: Reply with quote
Hello,

I've found another problem with DDBI, which might seem related to the topic above.

As I said above it is only possible to access the first two fieldnames. I had a table with only two columns (both int) and with million entries. So I thought I could use fieldnames as key to access the values. There is no problem iterating through all of the entries, but as soon as I add values from the Row results into a hashmap (either Tango hashmap or native hashmap) I get errors, that I can't access the value by the fieldname anymore. My thought was that the hashmap is overriding the Row array.

The strange thing is that this happens in curious ways. I get different results (number of iterations till the error occurs), if I change the number of entries of the results set (SQL limit). For example the error occurs after
9000 iterations in a foreach loop when the limit is set to 100, but I it iterates 3 million times, when I use 10.000 as limit for the result set.

any ideas? does this help to debug the above problem?

thanks
Lars

Last edited by lars_kirchhoff on Thu Jul 24, 2008 5:58 am; edited 1 time in total
Back to top
View user's profile Send private message
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway
PostPosted: Thu Jul 24, 2008 5:07 am    Post subject: Reply with quote
Hi Lars, can you post a minimal example that exposes this? Also, what is your system specs? OS, processor, etc
Back to top
View user's profile Send private message
lars_kirchhoff



Joined: 27 May 2008
Posts: 9
PostPosted: Thu Jul 24, 2008 6:50 am    Post subject: Reply with quote
Here is the minimal example (well its not that minimal Wink):

Code:

module ddbi_hashmap_test;

import dbi.mysql.MysqlDatabase;
import dbi.Row;
import tango.io.Stdout;

private import Integer = tango.text.convert.Integer;
private import tango.util.container.Container;
private import hm = tango.util.container.HashMap;

struct conn {
   uint id1;
   uint id2;
}

alias hm.HashMap!(conn, uint, Container.hash, Container.reap, Container.Collect) myHashMap;


int _create(uint limit)
{
   uint max_id = 500000;
   uint start_offset = 0;
   bool is_last = true;

   conn c;

   // database configuration
   char[] dbhost       = "";
   char[] dbname       = "";
   char[] dbusername    = "";
   char[] dbpw       = "";

   MysqlDatabase db = new MysqlDatabase();
   db.connect("host=" ~ dbhost~ ";dbname=" ~ dbname, dbusername, dbpw);

   myHashMap HM = new myHashMap;

   while (is_last) {

      Row[] entries = _getData(start_offset, start_offset + limit, db);

      if (entries.length > 0) {
         for (int i=0; i<entries.length; ++i) {
            c.id1 = Integer.parse(entries[i]["id1"]);
            c.id2 = Integer.parse(entries[i][0]);
            Stdout.formatln("Id 1: {}\t Id 2: {}", c.id1, c.id2);
            HM[c] = 1;
         }

         if (start_offset >= max_id) {
               is_last = false;
           }

           start_offset += limit;
      }
   }

   return 0;
}

Row[] _getData(uint start, uint end, MysqlDatabase db)
{   
   char[] sth = "   SELECT id1, id2 FROM data
                 WHERE id2 >= " ~ Integer.toString(start) ~ "
                 AND id2 < " ~ Integer.toString(end) ~ "
                 ORDER BY id2 ASC";

   return db.queryFetchAll(sth);
}



int main (char[][] args)
{
   uint limit;

   if (args.length > 1) {
      limit = Integer.parse(args[1]);
   }

   _create(limit);
    return 0;
}


With this code and a working database connection, I could reproduce the error. After several iterations I got this error:

Quote:
dbi.DBIException.DBIException: DBIException: The name 'id1' is not a valid index.


And the number of iterations I can do before the error occurs is depending on the limit I choose.

BTW, I get the error instantly when I use

Code:
alias hm.HashMap!(conn, uint, Container.hash, Container.reap, Container.Chunk) myHashMap;


as HashMap configuration.

But maybe I just produced buggy D code..

System configuration:
CPU: Intel(R) Xeon(R) CPU 2,4 Ghz
MEM: 4GB
OS: Linux version 2.6.16.13-4-bigsmp

thanks anyway
Lars
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic     Forum Index -> DDBI All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group