View previous topic :: View next topic |
Author |
Message |
uwe
Joined: 05 Apr 2005 Posts: 34 Location: Stuttgart, Germany
|
Posted: Fri May 20, 2005 7:03 am Post subject: Bug in mango.io.utf? |
|
|
Hi Kris,
during my work on the UTF converters i discovered a bug in your converter (basically because i copied it and modified then :) ). In the UTF-16 to UTF-8 converter you have to replace 0x8000 by 0x0800 (that value is the first to be encoded in 3 bytes).
Ciao
uwe |
|
Back to top |
|
|
kris
Joined: 27 Mar 2004 Posts: 1494 Location: South Pacific
|
Posted: Fri May 20, 2005 10:07 am Post subject: Re: Bug in mango.io.utf? |
|
|
uwe wrote: | Hi Kris,
during my work on the UTF converters i discovered a bug in your converter (basically because i copied it and modified then ). In the UTF-16 to UTF-8 converter you have to replace 0x8000 by 0x0800 (that value is the first to be encoded in 3 bytes).
Ciao
uwe |
Thanks, uwe; I'll check it out.
I noticed a couple in your versions also, though you may have fixed 'em by now. From memory:
~ in String.toUtf8, the realloc does not do a complete job. The base pointer was not updated, I recall.
~ in the newer conversion module, there's a similar conflict going on with the starting address of the local pointer; sometimes it is zero-based and sometimes it is offset by the 'index' argument passed in (second arg). The inconsistency will break in a boundary condition.
Sorry this is short on detail ~ I don't have the code in front of me.
If you'd give your code a non-viral license, then I'd use it in Mango (it is kind of annoying when people take open-source code and produce a modified version with a more restrictive license -- it destroys the notion of rolling changes back into the original code, which surely negates the spirit of open source). In fact, what you appear to be doing with your String class and ICU is strikingly similar to what mango.icu provides.
What do you think about combining efforts instead? I mean, Mango is about to adopt a Container package, and perhaps your design is a better approach than, say, MinTL? |
|
Back to top |
|
|
teqdruid
Joined: 11 May 2004 Posts: 390 Location: UMD
|
Posted: Fri May 20, 2005 11:48 pm Post subject: OT: Containers |
|
|
Quote: | Mango is about to adopt a Container package |
Wha??? I didn't hear a word about this... Given I've been a bit busy lately, but when is this happening? This is great news!
I've been waiting for Ben to finish his interface-type stuff for MinTL, but based on how Mango is currently structured, I'm betting it's built with a good class hierarchy.
So? When's it happening? If not soon, can I (we) have a sneak peak?
Alright... where's the "anxious" emoticon thingy? |
|
Back to top |
|
|
kris
Joined: 27 Mar 2004 Posts: 1494 Location: South Pacific
|
Posted: Fri May 20, 2005 11:58 pm Post subject: |
|
|
Why didn't you say so before?
It's really just been a case of priority, and I've been waiting for Ben too! I want to keep the options open though, and perhaps Indigo has something nice that we can adopt, or merge with? |
|
Back to top |
|
|
teqdruid
Joined: 11 May 2004 Posts: 390 Location: UMD
|
Posted: Sat May 21, 2005 12:11 am Post subject: |
|
|
I don't really care where it comes from, but I need a good Containers API with a good class hierarchy. If it's in Mango, all the easier for me, plus it'll probably have the same great hierarchy that the rest of Mango has.
I have't looked at either Indigo, or DTL (which I doubt even compiles at this point) but the little I've seen of MinTL doesn't make me think that it would be a good candidate for inclusion in Mango- unless Ben is making some pretty radical changes.
It seemed more like a C-style API to me. Almost no inheritance, and really boils down to a collection of structs and associated functions. I'm not saying this is a bad thing- I'm betting Ben gets a lot of speed out of it, but it just didn't seem in the Mango style... I'll reserve final judgement until I see some of his latest stuff, however.
I'm hoping it won't take too long. I just finished finals today, so I've now got some free time (famous last words) to get back to work on my project.
I'll take a look at Indigo soon. |
|
Back to top |
|
|
kris
Joined: 27 Mar 2004 Posts: 1494 Location: South Pacific
|
Posted: Sat May 21, 2005 1:45 am Post subject: |
|
|
teqdruid wrote: | I'm hoping it won't take too long. I just finished finals today, so I've now got some free time (famous last words) to get back to work on my project. |
Congratulations dude!
And welcome back, too. |
|
Back to top |
|
|
uwe
Joined: 05 Apr 2005 Posts: 34 Location: Stuttgart, Germany
|
Posted: Sun May 22, 2005 11:53 pm Post subject: |
|
|
Sorry i took so long for my answer, i simply forgot to check back :(
I have already discovered the first bug you spoke about. But i am not so sure about the second... The conversion module has passed all unit tests after some debugging, but i'll look into it again, thanks a lot.
I am not sure what you mean with "viral license". Do you mean you can't take something from my code back into your project because you would have to change your license?? Well, no prob, i was not thinking much about licensing issues (all that stuff sometimes annoys me), just taking the "popular" one i know. Can be changed, you can simply take my code, distribute it under your license and write somewhere "uwe helped me" or similar :)
Sure you can use Indigo in Mango. I was dissatisfied about the String class either (that's why i did not write it to perfection), mostly because of the dumb ICU porting that just produces bloat, and was already done by you. I did not want to be dependant on ICU _and_ Mango, that's all. If a string class weren't so damn important for almost any program, i would have abandoned it.
Ciao
uwe |
|
Back to top |
|
|
kris
Joined: 27 Mar 2004 Posts: 1494 Location: South Pacific
|
Posted: Mon May 23, 2005 1:05 am Post subject: |
|
|
uwe wrote: | I am not sure what you mean with "viral license". |
Oh, that's a general term for a GPL-type license. As you noted, adoption of the code tends to require adoption of the license. Thanks for clarifying!
uwe wrote: | I did not want to be dependant on ICU _and_ Mango, that's all. If a string class weren't so damn important for almost any program, i would have abandoned it. |
Yeah; I know what you mean, though it would be beneficial to users if the two string classes were equivalent at the interface level. BTW, mango.icu is completely standalone when you compile it with as is the mango.log package.
On a different note ~ if you're going to continue expanding the Indigo container facilities, are you making them thread-safe? If so, I'd be interested in them. Also, you may find mango.cache.HashMap useful for Indigo, since it's a port of Doug Lea's non-blocking thread-safe hashmap. Prof Lea writes very high quality code, and is a prolific writer of invaluable thread-oriented utilities. Ben Hinkle's Locks lib is a straight port of Doug's Java library ~ effectively the one that now ships with Java 1.5 |
|
Back to top |
|
|
uwe
Joined: 05 Apr 2005 Posts: 34 Location: Stuttgart, Germany
|
Posted: Mon May 23, 2005 4:02 am Post subject: |
|
|
As i said, if you need something of the Indigo code (or the utf conversion package), just take it. No need to include the license as well. Something like "other authors: uwe" would be nice. I'm always happy if i can help.
I have already looked at your hashmap, and will perhaps use it as an implementation draft. I do not plan to make Indigo thread-safe in the near future, as the Qt containers are not, too (ok, they are somehow thread-safe, but you cannot access the same container from two threads, just copy it thread-safely). But i will look over Doug Lea's code, thanks for the hint.
By the way, did you need thread-safe containers often in your projects? I never needed one yet, and i program for quite a long time (ok, did not write so much threaded code either). Just once i had a very large and complex data structure that had to be thread-safe, but that was really a more complicated issue than solvable with a thread-safe vector/list/whatever.
Ciao
uwe |
|
Back to top |
|
|
kris
Joined: 27 Mar 2004 Posts: 1494 Location: South Pacific
|
Posted: Mon May 23, 2005 11:32 am Post subject: |
|
|
uwe wrote: | As i said, if you need something of the Indigo code (or the utf conversion package), just take it. No need to include the license as well. Something like "other authors: uwe" would be nice. I'm always happy if i can help.
I have already looked at your hashmap, and will perhaps use it as an implementation draft. I do not plan to make Indigo thread-safe in the near future, as the Qt containers are not, too (ok, they are somehow thread-safe, but you cannot access the same container from two threads, just copy it thread-safely). But i will look over Doug Lea's code, thanks for the hint.
By the way, did you need thread-safe containers often in your projects? I never needed one yet, and i program for quite a long time (ok, did not write so much threaded code either). Just once i had a very large and complex data structure that had to be thread-safe, but that was really a more complicated issue than solvable with a thread-safe vector/list/whatever. |
The mango.http.server and mango.cluster packages are quite heavily mulit-threaded, and that's where the need for an efficient thread-aware container came me up (I like to avoid synch if possible; I'm sure you can sympathize). Those packages make heavy use of thread-locals to avoid shared resources for the most part.
That aside; the industry has been heading towards multi-core devices for a while now, and I suspect it'll continue that way for at least the next decade or two. I'm sure you're aware of commercial designs like Cell, and Niagara (the latter has 32 hardware-supported threads). I mentioned a long time ago that it'd be nice for D to investigate some of the Occam constructs for parrallel programming ~ things like concurrent loops and so on; Threads are good for now though
BTW; some desktop-style packages are making the transition by building a producer/consumer aspect into the software ~ that way they can pipeline heavy processing across two threads, and take advantage of extra hardware without the added complexity of a fully threaded design. Makes a lot of sense.
Debugging multi-threaded software is still the nightmare it always was; but it's made a lot easier by having pre-tested thread-aware containers, and by the existence of libraries such as Doug Lea's utilities. |
|
Back to top |
|
|
uwe
Joined: 05 Apr 2005 Posts: 34 Location: Stuttgart, Germany
|
Posted: Mon May 23, 2005 1:55 pm Post subject: |
|
|
I looked at Doug Lea's stuff, especially at the containers. But there were no concurrent lists/arrays, just copy-on-write versions, and versions wrapped in synchronizations (which can be easily realized in D by synchronized{} statements where necessary). Thus the Vector and List/LinkedList can stay as they are, String anyways. As the Map is a linked list too, it cannot be made thread-safe as well.
But i will try to do the trick with the Hash, if it does not impact on performance and simplicity too much. As you said, Ben Hinkle ported the stuff already. Much as i would be pleased to see Indigo in Mango, the aim for Indigo is to provide simple but high-performance containers for people that want to write productivity apps in D without understanding the difficult design of yet another template library.
Ciao
uwe |
|
Back to top |
|
|
kris
Joined: 27 Mar 2004 Posts: 1494 Location: South Pacific
|
Posted: Mon May 23, 2005 2:24 pm Post subject: |
|
|
uwe wrote: | But i will try to do the trick with the Hash, if it does not impact on performance and simplicity too much. As you said, Ben Hinkle ported the stuff already. Much as i would be pleased to see Indigo in Mango, the aim for Indigo is to provide simple but high-performance containers for people that want to write productivity apps in D without understanding the difficult design of yet another template library. |
Just to be clear, Ben has a seperate library called Locks (outside of MinTL). It is class based, since it's a straight port of Lea's synchronization lib.
And I certainly agree regarding the difficulties inherent in templated designs. They're not for the faint-of-heart, and not easy to pick up. Plus, they can be truly hideous to debug once deployed (just like enormous macros; the debugger gets very confused). It's unlikely anyone will see much of that in Mango, either. |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|