FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Library Chaos
Goto page 1, 2  Next
 
Post new topic   Reply to topic     Forum Index -> Ares
View previous topic :: View next topic  
Author Message
lgoss007



Joined: 13 Oct 2005
Posts: 20

PostPosted: Thu Oct 13, 2005 4:27 pm    Post subject: Library Chaos Reply with quote

Hello all. I'm a C++ programmer by night, but for my job most of my coding has to be in C#. I've come to like some of the simplicities of C#/.NET, but I've always preferred system languages that are general purpose (allow many styles of programming). But what C# really shines in is it's libraries.

Well I've been wandering around the various D sites trying to get caught up on all that is going on in the language and in the library realm, and it appears to be segregated and sort of chaotic. A lot of projects seem to adapt their own libraries (mango for instance with it's FileSystem, Sockets, etc.) and their doesn't seem to be one standard library (which simplifies things greatly).

So... what direction is Ares heading? Are you planning in the future to have an all inclusive library (database support, imaging, etc.)? Or will the library encompass a few select standards?

I ask because I would like to help out somewhere with the Linux part of some library, especially in the area of 3d graphics, such as OpenGL, X Windows, OpenSceneGraph, etc. However I'd rather not create a whole seperate library of my own, I'd prefer most of my work to be part of a "standard" library (even if the standard is seperated out into seperate parts).

Anyways, I had to just get all of that out. I'm just really interested in doing some OpenGL stuff in D and I'd like it to be in a standard library (after all OpenGL is a standard Very Happy ). Also I think Vectors, Matricies, and a whole slew of 3D libraries and math would be nice to have in the library. Ok, I'll stop now, thanks for listening.
Back to top
View user's profile Send private message Send e-mail AIM Address Yahoo Messenger
sean



Joined: 24 Jun 2004
Posts: 609
Location: Bay Area, CA

PostPosted: Fri Oct 14, 2005 9:14 am    Post subject: Re: Library Chaos Reply with quote

lgoss007 wrote:
Hello all. I'm a C++ programmer by night, but for my job most of my coding has to be in C#. I've come to like some of the simplicities of C#/.NET, but I've always preferred system languages that are general purpose (allow many styles of programming). But what C# really shines in is it's libraries.

Well I've been wandering around the various D sites trying to get caught up on all that is going on in the language and in the library realm, and it appears to be segregated and sort of chaotic. A lot of projects seem to adapt their own libraries (mango for instance with it's FileSystem, Sockets, etc.) and their doesn't seem to be one standard library (which simplifies things greatly).

DMD includes Phobos, which is the default standard library. Not as complete as the .NET or Java libraries perhaps, but it's there.
Quote:
So... what direction is Ares heading? Are you planning in the future to have an all inclusive library (database support, imaging, etc.)? Or will the library encompass a few select standards?

I intend for Ares to be a very modular library. Its core will be pretty basic--perhaps along the lines of the C++ standard library--but it will have support for extensions to support more specialized work. ICU is a likely candidate for internationalization, etc. Basically, I want to have a set of essential core components that are used in most every project and then interface-consistent addons that a user can select as needed. An embedded systems programmer doesn't have any interest in much beyond the basic, for example.
Quote:
I ask because I would like to help out somewhere with the Linux part of some library, especially in the area of 3d graphics, such as OpenGL, X Windows, OpenSceneGraph, etc. However I'd rather not create a whole seperate library of my own, I'd prefer most of my work to be part of a "standard" library (even if the standard is seperated out into seperate parts).

Ares is still in its formative stages, so the interface guidelines and such are still evolving, but feel free to submit anything you'd like for inclusion. Keep in mind, however, that there are other projects in this area on DSource so you may be duplicating work Smile
Back to top
View user's profile Send private message
lgoss007



Joined: 13 Oct 2005
Posts: 20

PostPosted: Fri Oct 14, 2005 12:31 pm    Post subject: Reply with quote

Quote:
...I want to have set of essential core components that are used in most every project and then interface-consistent addons that a user can select as needed. An embedded systems programmer doesn't have any interest in much beyond the basic, for example.


I agree, you should be able to select what you need.

Quote:
Keep in mind, however, that there are other projects in this area on DSource so you may be duplicating work


This is the problem. As you mentioned above, you want "interface-consistent addons". However, since there are a bunch of different projects on dsource (some which appear not to have much life in them), each of them are working on their own stuff. Most likely they are building their own libraries which will be duplicated because someone wants only a piece of what they've writtin (a small part of their custom library). This makes it harder to find libraries and harder to submit or know where to submit code. I don't want to see it end up like the C++ libraries, there are so many different ones that people built a library for just their project, and didn't think about the big picture. The boost library is a good solution (partially) to fix that, but that doesn't appear to be what's going on here at dsource. It's more like:

Quote:
Bob: I'm looking for OpenGL libraries.
Mike: It's in project A (some GTK+ project).
Bob: Ok, well I want to use OpenGL with X Windows.
Mike: The X libraries are in project B.
Bob: What about sound libraries?
Mike: Project G
...


I want a central library much like boost, however I want the library to include much more than boost (boost is alright, but it's nothing compared to the java and .net libraries).

A central library for D would rock. Then you could have:
sys.core
sys.gl
sys.international
...

And each specialized work could be a sub-project, but it would all be a part of the D library. Of course individuals could pick and choose what they want, like the embedded system programmer (of course OpenGL ES is targeted at embedded systems, Very Happy )
Back to top
View user's profile Send private message Send e-mail AIM Address Yahoo Messenger
sean



Joined: 24 Jun 2004
Posts: 609
Location: Bay Area, CA

PostPosted: Fri Oct 14, 2005 1:58 pm    Post subject: Reply with quote

lgoss007 wrote:
I want a central library much like boost, however I want the library to include much more than boost (boost is alright, but it's nothing compared to the java and .net libraries).

A central library for D would rock. Then you could have:
sys.core
sys.gl
sys.international
...

And each specialized work could be a sub-project, but it would all be a part of the D library. Of course individuals could pick and choose what they want, like the embedded system programmer (of course OpenGL ES is targeted at embedded systems, Very Happy )

I agree completely. It would also be very nice if D had a pragma to auto-include libraries during the build process, though I assume Derek's Build project fits the bill somewhat.
Back to top
View user's profile Send private message
AgentOrange



Joined: 21 Jul 2005
Posts: 61

PostPosted: Fri Oct 14, 2005 2:48 pm    Post subject: Reply with quote

Yah that is definitely a goal, to have a unified project tree. Im not so worried about it at this stage tho, because so many of the projects are so young. Ive found it personaly easier to grab the code and put it in my project then *I* can change it as I need, and not have to deal with another project. I think we do need to start talking about where we want to ultimately go with this, because we could do some incredibly cool stuff. Im working on some tools like a debugger and hoping to help pragma with getting dynamic libs going right now. But I think in time the projects will mature and we will be able to do some fantastic stuff.
Back to top
View user's profile Send private message
AgentOrange



Joined: 21 Jul 2005
Posts: 61

PostPosted: Fri Oct 14, 2005 2:53 pm    Post subject: Reply with quote

sean wrote:
I agree completely. It would also be very nice if D had a pragma to auto-include libraries during the build process, though I assume Derek's Build project fits the bill somewhat.


Yeah and we really could use a tool to manage the high level packages. Im thinking of something that is a cross between a package manager and a source repository. Maybe even a special subversion client would do it.
Back to top
View user's profile Send private message
lgoss007



Joined: 13 Oct 2005
Posts: 20

PostPosted: Fri Oct 14, 2005 3:45 pm    Post subject: Reply with quote

Sounds good! Yeah the Build tool is pretty nice, and dynamic libs are definately needed. I could also use a good IDE too. Guess I'll just kinda keep my own local library of things then till Ares and other projects start growing a little more then.
Back to top
View user's profile Send private message Send e-mail AIM Address Yahoo Messenger
thedracle



Joined: 25 Dec 2005
Posts: 11

PostPosted: Thu Dec 29, 2005 7:37 pm    Post subject: D-BOOST Reply with quote

I've experienced a lot of the same segmentation described above among libraries in D, but a lot of this can be attributed to the general youth of the language--- and the fact that the standard library, and the compiler, are still undergoing developmental changes.

Java wasn't really a open-development platform to begin with (although this has changed somewhat with the java community process), Sun did an excelent job of creating an extensive and complete standard library at launch, despite having to make a great deal of changes (new collections framework, new UI API, etc..) in the 1.2 version. C# essentially implemented and copied a lot of the funcitonality provided by the Java standard API, and also included a lot of the useful features of previous windows libraries. I would say, as a system's development language, D is probably the best I've seen--- and it shows promise for being as good as Java or C# for user-mode applications. Having a library as powerful as that in Java and C#, and maybe one as pragmatic, simple, and reusable as the standard libraries in Ocaml/Ruby (because of the presence of delegates), would make it very appealing. I've already experimented with implementing Files and sockets in a fashion similar to Ruby, using function delegates for iteration:

i.e:

Code:
File myFile = new File("test.txt", "rw+");
void readLine(char[] nextLine)
{
   //Process next line.
   printf("?.*s\n", nextLine);
}
myFile.eachLine(&readLine);


Would iterate over every line in the file, using the given delegate to proccess the data.

A TCPProxy could be implemented similarly, hiding most of the details of handing the sockets, looping, and threading:

i.e:
Code:

int clientDelegate(char[] nextBuf)
{
   // Perform logging, analyzing, filtering, of the given packet.

   // Return false to block forwarding this packet to the server.
   return forwardToServer;
}
int serverDelegate(char[] nextBuf)
{
   // Perform logging, analyzing, filtering, of the given packet.

   // Return false to block forwarding this packet to the client.
   return forwardToClient;
}
TCPProxy mProxy = new TCPProxy;
mProxy.connect("hostadd.nowhere.com", hostport, listenport,  &clientDelegate, &serverDelegate);


As could a TCPSocket itself, which would take a server IP, port, and a delegate to handle the input for each line. D opens up the possibility of adopting much of the more simpler and elegant coding styles seen in languages like smalltalk, for use in system level programming.

It seems unfair to pick on the various libraries springing up here and there, BOOST is a compilation of some libraries that began in a similar fashion to meet needs unmet by the standard library in C++. The D standard library is very much incomplete when compared with that of Java, or C#, or even C++--- and I find the quality that most redeems this fact is that the language itself happens to implement a lot of the functionality that a great deal of these libraries tend to provide for. I can see the possibility of a much cleaner, easier to use, and easier to read, library being implemented in D, by not following the exact examples of C#, and Java--- but instead other more modern languages.

-Jason Thomas.
Back to top
View user's profile Send private message
kris



Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

PostPosted: Thu Dec 29, 2005 8:21 pm    Post subject: Reply with quote

D can go one better vis-a-vis iterators. Instead of explicitly using delegates, one can use foreach() in conjunction with opApply() to get the same thing ~ potentially cleaner in many circumstances. The Mango library applies this in a number of areas, including the latest tokenizers (such as LineToken).

For those who don't already know: in the case of the TCPProxy (above), where more than one delegate is used, it's often better to use either an abstract class or an interface instead ~ if you wish to keep everything together in a nice clean bundle. FWIW: Interface methods can be treated directly as delegates too ... I suspect the implementation of Interface is based upon a list of delegate entries.

That aside, I agree in full with the gist of your post: D has the potential to make some really easy-to-use, extensible, libraries.
Back to top
View user's profile Send private message
lgoss007



Joined: 13 Oct 2005
Posts: 20

PostPosted: Thu Dec 29, 2005 8:57 pm    Post subject: Reply with quote

Quote:
It seems unfair to pick on the various libraries springing up here and there, BOOST is a compilation of some libraries that began in a similar fashion to meet needs unmet by the standard library in C++. The D standard library is very much incomplete when compared with that of Java, or C#, or even C++--- and I find the quality that most redeems this fact is that the language itself happens to implement a lot of the functionality that a great deal of these libraries tend to provide for.


Yes the language implements a lot of the functionality and that is nice. However I feel the need to pick on the D library because Java and C# both have nice fairly well featured libraries, but D doesn't seem to be heading that way (I don't mention C++ because it's library lags quite a bit behind). In the newsgroup there was talk about sort of copying the Java or C# library, but most seemed to say that D could do better. Which is fine... but I fear things will be segregated, much like in C++ now. I would just like some plan, layout, design, something to follow to implement such a huge library, as I make no claim nor do I think I have the experience to be a library designer (at least not for the size of a library like Java or C#, which is what I really want D to have).

I just like the idea of a central library because: it's easier to find the library you need, you know the library is good because many people are working on them from many different fields (instead of all making their own where they may be an expert in their field, but they may not consider another field), and the library is consistent (which makes it much easier to use).
Back to top
View user's profile Send private message Send e-mail AIM Address Yahoo Messenger
kylefurlong



Joined: 16 Nov 2005
Posts: 29

PostPosted: Fri Dec 30, 2005 3:20 am    Post subject: Reply with quote

Edit: I just reread this and I think I got a little off topic. You can treat this as a rant. Embarassed

An informative anecdote is my own situation.

I wrote an app in C# to help me pick stocks, do statistics etc. I ran into performance issues as the dataset got large, so I went shopping for another language. Searching on this great big web of ours I found D. It seemed to be the solution for me: all the performance of C/C++, and the modern language features that I had come to love in Java and C# land, plus, thoughtful new features that I HADNT seen before which made my life easier (the meta programming system, slicing, etc.). D the language was just what I needed.

Coming from a C#/Java background though, I found that I could not be productive. No IDE, no (easy, with inspection) debugging, no drop in libraries (assemblies, jar files). What libraries I could find were either incomplete, incompatible, or unreliable. Basically I found, in my view, a shoestring and spit operation. D the infrastructure was a complete failure.

So... from my perspective, what needs improving is not the language, but the infrastructure surrounding the language, meaning the standard library, and all those other things I mentioned.

Anyways, I guess what I am talking about is a unification of effort, like the origionator was speaking of. We have Posiedon, DDL, Mango, Ares, DParser, Dbug, and many other projects which are trying to fill in the missing peices of a complete developing environment. Why cant these together become a standard for D? Posiedon + DParser + DBug = IDE.

Anyways my two cents at 1 am.
Back to top
View user's profile Send private message
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway

PostPosted: Fri Dec 30, 2005 4:06 am    Post subject: Reply with quote

kylefurlong wrote:
Why cant these together become a standard for D? Posiedon + DParser + DBug = IDE.


I'll use Poseidon if it gets Vim-semantics and bindings, and becomes a KPart for direct integration everywhere in KDE (and this more or less requires DWT to have a Qt-backend, which I suppose is rather difficult Wink Seriously, I agree with the library stuff, I always did (and I try to help with DDL at the moment), but having the complete IDE won't make me any more productive than having Vim right now. that's probably just me, though.
Back to top
View user's profile Send private message
thedracle



Joined: 25 Dec 2005
Posts: 11

PostPosted: Fri Dec 30, 2005 6:17 pm    Post subject: Unified Library Reply with quote

From kris:
"D can go one better vis-a-vis iterators. Instead of explicitly using delegates, one can use foreach() in conjunction with opApply() to get the same thing ~ potentially cleaner in many circumstances. The Mango library applies this in a number of areas, including the latest tokenizers (such as LineToken)."

I would say using iterator syntax is questionably better, it eliminates having to define a named delegate (you're kindof using an anonymous delegate), but it does so at the expense of eliminating a potential return value (for instance with collect), reusing that delegate for multiple streams, and a clear English title for precisely the operation you're performing. For instance: you may want to return a byte, or a string value, in multiple ways, where an iterator would determine the value to use only by the return type.

For instance:

Code:

class ExampleIterator
{
   void onApply(int delegate(char[] str) uCode)
   {
      char[] val;
      int result = 0;
      // Forward data to user.
      if((result = ((val = getSomeData()) == null)?0:1))
         uCode(val);
      return result; // Go to next element.
   }

    // Now handle another type of output that also returns a string?
}


I intentionally made that statement as ugly as possible simply for the psychological effect of making you cringe at seeing it--- with the desired effect of forwarding my own agenda.

I would say, in general, sticking to the principle of least surprise is the best idea. From the eachLine identifier, it's pretty easy to tell precisely what the function is going to be doing. Adding a eachLineCRLF makes it even clearer that the original method returns a stripped string, and allows different functionality for the same return type. Most importantly it sticks to a convention of using callbacks and precisely named methods for doing operations like collect, and operations like file/socket iteration.

foreach I feel is a bit superfluous, and is syntax sugar--- but the fact that it's extensible in D, and not pure compiler magic, makes it a lot better than a lot of other languages I've seen.

A more pragmatic library might avoid using foreach in preference for higher order functions and callbacks--- but the subject is debatable, the cleanliness it adds may be more worthwhile.


Kris:
"For those who don't already know: in the case of the TCPProxy (above), where more than one delegate is used, it's often better to use either an abstract class or an interface instead ~ if you wish to keep everything together in a nice clean bundle. FWIW: Interface methods can be treated directly as delegates too ... I suspect the implementation of Interface is based upon a list of delegate entries."

I'd like to see an example of this before I form an opinion about it. I like the fact that anonymous classes in Java are able to bundle together related functions into a cohesive entity (even though they're ugly in practice)--- this is actually one of the major drawbacks of functional languages and smalltalk's code blocks. If D can do the same thing, but with complete access to the stack above it, in a clean manner (and hopefully without a overhead penalty of heap allocation?)--- I would agree with you on this.

lgoss007:
"However I feel the need to pick on the D library because Java and C# both have nice fairly well featured libraries, but D doesn't seem to be heading that way."

I definitely agree that D definitely needs a well structured, cohesive, and more complete, standard library. But...

"In the newsgroup there was talk about sort of copying the Java or C# library, but most seemed to say that D could do better."

This is exactly what I think, I think D could do it 'much' better, and differently.

And I was simply pointing out that multiple projects springing up to fill these gaps really aren't the problem, I gave the example of BOOST to demonstrate how such libraries can be adopted by a standard library. The fact that there isn't yet a cohesive uniting entity, like the Java Community Process, to create and adopt these projects into a standard library, is a problem. This is especially true since it's likely these projects are following divergent coding philosophies and styles, and the resulting integrated library will definitely not follow the principle of least surprise.

kylefurlong:
There are plenty of extensible IDEs out there, we definitely need a project for adapting D to a more productive application environment. DWT and Poseidon are making significant steps in this direction. But I agree with larsivi that the IDE is really secondary to the language itself. I find myself enormously more productive, without an IDE, with a well designed set libraries, and in a good language (Like Java, Ruby), than in a frankensteined language with an excellent IDE (Visual C++).

Thanks for the comments guys, I can tell D has a very active and intelligent users community.

-Jason Thomas.
Back to top
View user's profile Send private message
sean



Joined: 24 Jun 2004
Posts: 609
Location: Bay Area, CA

PostPosted: Fri Jan 06, 2006 12:24 am    Post subject: Re: Unified Library Reply with quote

thedracle wrote:
I would say using iterator syntax is questionably better, it eliminates having to define a named delegate (you're kindof using an anonymous delegate), but it does so at the expense of eliminating a potential return value (for instance with collect), reusing that delegate for multiple streams, and a clear English title for precisely the operation you're performing. For instance: you may want to return a byte, or a string value, in multiple ways, where an iterator would determine the value to use only by the return type.

While I do have issues with the C++ standard library, I do very much like the algorithm-based approach it's built around. Most things can be represented as an iterator and problems are solved via algorithm composition. Sometimes the concept is stretched a bit thin--input iterators, for example--but all in all it works quite well. I have two major complaints about foreach. First, foreach only allows one range to be operated on at a time, and second, it's not really possible to stop midway and continue later on. And both are fairly common in code I write. That said, without implicit template instantiation, using iterators in D is kind of awkward.
Quote:
I would say, in general, sticking to the principle of least surprise is the best idea. From the eachLine identifier, it's pretty easy to tell precisely what the function is going to be doing. Adding a eachLineCRLF makes it even clearer that the original method returns a stripped string, and allows different functionality for the same return type. Most importantly it sticks to a convention of using callbacks and precisely named methods for doing operations like collect, and operations like file/socket iteration.

It's an interesting idea, and one that extends well. I'm trying to find some time to think about IO for Ares and will keep it in mind when I do.
Quote:
foreach I feel is a bit superfluous, and is syntax sugar--- but the fact that it's extensible in D, and not pure compiler magic, makes it a lot better than a lot of other languages I've seen.

Agreed. And it works for a good number of common cases, particularly in combination with slicing.

Quote:
I definitely agree that D definitely needs a well structured, cohesive, and more complete, standard library. But...

"In the newsgroup there was talk about sort of copying the Java or C# library, but most seemed to say that D could do better."

This is exactly what I think, I think D could do it 'much' better, and differently.

I very much agree. And I think it would help to find a 'theme' of sorts for the standard library, much as iterators/algorithms are the theme for C++. If you have any suggestions, please share them. I haven't yet been completely sold on the idea of foreach, which seems to be a defining feature of D.
Quote:
And I was simply pointing out that multiple projects springing up to fill these gaps really aren't the problem, I gave the example of BOOST to demonstrate how such libraries can be adopted by a standard library. The fact that there isn't yet a cohesive uniting entity, like the Java Community Process, to create and adopt these projects into a standard library, is a problem. This is especially true since it's likely these projects are following divergent coding philosophies and styles, and the resulting integrated library will definitely not follow the principle of least surprise.

I had originally hoped that Ares would fill the gap between the community and the powers that be--kind of a more structured version of the Boost project. But official feedback is simply so difficult to obtain that it seemed more reasonable to reposition Ares as a replacement standard library. It is my hope that Ares will ultimately become the de facto standard simply through merit and sheer inertia. At this point, the real obstacles are free time and community participation.
Back to top
View user's profile Send private message
thedracle



Joined: 25 Dec 2005
Posts: 11

PostPosted: Fri Jan 06, 2006 6:52 pm    Post subject: Perfect Library. Reply with quote

sean:
"It's an interesting idea, and one that extends well. I'm trying to find some time to think about IO for Ares and will keep it in mind when I do."

I think using delegates in a standard way for File I/O, and socket I/O, is clearly demonstrated in Smalltalk/Ruby. I think looking there to begin with for examples would be a good idea.

Normal File I/O, Socket I/O can be implemented pretty simply, i.e:

Code:

File myFile = new File("filename.dat", "rw+");
TCPSocket mySocket = new TCPSocket("bob.nowhere.com", 8020);
bit processLine(char[] nextLine)
{
   char[][] matches;
   if((matches = nextLine.matches(someRegExp) != null)
   {
      // Do something.
   }
}
myFile.eachLine(&processLine);
mySocket.eachLine(&processLine);


Basically the File object, and the Socket object, both implement an underlying IO interface that has the eachLine, eachByte, etc... functions. It seems IO and collections share this basic interface as well--- maybe there exists some base type that both IO and collections can be derived from?

I've thought about it a bit, and having foreach on top of this could work pretty well, you could have a onApply wrapper function for the expected default function (the one that we expect people to use most often)--- and people can simply decide which style they want to use.

Another big question involves the lack of a standard set of collections in D. Dynamic arrays do have a great deal of functionality that a standard collection library would implement, but not all of that functionality. I think standard collection types, such as Linked Lists, Red-Black Binary Trees, Sets with unique elements, synchronized collections Vs. unsynchronized collections, should be implemented in a standard library. A default Collection class may have methods like 'sort', 'collect', 'each', 'isempty', etc..

Object wrappers should also exist for standard data-types--- but we'll likely want to use generics to allow use of primitives in these collections? I'm not very fond with generics in general, but this is coming from experience in C++--- I'm not sure if they produce any more readable code, or errors, in D?

Gotta work now, but thanks for the reply. It'd be really interesting to get a collaboration together to just get a list of APIs and features that people think the standard library ought to have.

-Jason Thomas.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic     Forum Index -> Ares All times are GMT - 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group