FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

library organization

 
Post new topic   Reply to topic     Forum Index -> Ares
View previous topic :: View next topic  
Author Message
gabe



Joined: 25 Apr 2006
Posts: 4
Location: Chicago

PostPosted: Wed Apr 26, 2006 9:29 am    Post subject: library organization Reply with quote

hello,
I came across D a few months ago, but didn't really look very far into it. I happenned to stumble across it a few days ago, and so far, I'm quite impressed with the language. I've been burned by C and I'm tired of Java, and I think D has some real potential. However, I do have some serious concerns about the organization and structure of the libraries. I would love to see an open source version of the libraries, with a full stack implementation of a number of things, but in looking over several projects, one thing strikes me as particularly lacking between every version (particularly that of the original d library, phobos). First, as an OO language, it seems that it should be organized more heirarchically. Much of what I see in ares is quite flat, and would be difficult to extend after a certain point. I think it should be a priority to lower the threshold of entry for other programmers by introducing a common interface. I would highly suggest that the d community adopt a structure much like that found in the java language. Obviously, the d community could and should improve on java's mistakes (most notably the horrid io packages). As I understand it, other packages like Mango make use of a much better file system like this.

In any case, upon reflecting on the situation, I came up with my own version of what the d library should look like. I'd like to know what other people think. I'd just like to say that in so doing, I want to make it easier for d to become the commercial success I feel it could be (contract programming and inherent unit tests?? wow!). I think that the best part about this system of modularization is that the low level API and ASM code can be partitioned off and compiled separately depending upon the platform without ever affecting any of the other code. a flatter system like Ares doesn't seem to work that well for that kind of large-scale OO and procedural abstraction.

Code:

d/
  lang
    exception
    gc
    bitarray
    boxer
    concurrency
      thread
        locks
        ...
      monitor
        ...
      ...
    reflection
 
  extern
    c
      stdio
      math
      fenv
      process
      stdarg
      stddef
      stdlib
      string
      time
      ...
    c++
      ...
 
  data (collections/standard data structures)
    ArrayList
    Stack
    Queue
    LinkedList
    DoublyLinkedList
    CircularlyLinkedList
    BTree
    HashMap
    ...
   
  util
    compress (protocols for bzip, gzip, tar etc.)
    logging
    date
    random
   
  sys
    kernel (possibly system confgured build info for each platform?)
    memory
    ...

  os
    posix
    windows
      c
        ...
      com
        ...
      dotnet  (???)
    linux
      c
        ...
      kernel (???)
        ...
      userland (???)
    titan
    minix
    ...
   
  security
    crypt
      md5
      sha1
      ...
    dgp (a port of gnu pgp might be very helpful)
  math
    ieee
    ...
 
  thea (D language graphical system based on the qt framework)
    opengl
    window
    frame
      button
      ...
    ...
 
  text
    encoding
      unicode
      ansi
      codepage
    regex
    format
    ...
 
  sql
    dbi (database interface abstraction)
    vendor (specific vendor APIs)
    ...
   
  io
    file
    buffer
    pipe
    stream
    ...
 
  net
    socket
    server
      http
      udp
      ftp
    mail
      encoding
      protocols
    ident
      url
      uri
      ...
    ...
 
  xml
    parsers
    validators
    xpath
    xslt
    ...


You may have noticed that I put in an entry there for a gui framework called thea, for the greek goddess of sight. (I thought that since there's already all these mythical libraries around (Titan, Ares), why not go for one more?) It would port qt 4.1 into d. That would be a good deal more complicated than porting swt or some other framework, but the results would probably be much, much more interesting, as qt is (I think obviously) the better framework for graphics programming.
_________________
New to the land of D, but I like the look of the place.
-Gabe
Back to top
View user's profile Send private message
sean



Joined: 24 Jun 2004
Posts: 609
Location: Bay Area, CA

PostPosted: Wed Apr 26, 2006 10:05 am    Post subject: Reply with quote

I'll admit I'd considered a more heirarchical structure in the past, but decided against it at the time largely because I couldn't come up with a good design. I'll have to give it some thought, but overall, this seems fairly workable. I'm still not sure where algorithms (such as std.array) might go however, and 'extern' would have to be changed because it's a keyword in D. Another point worth noting is that Ares will be a modular library, and so the layout must be somewhat pluggable. This design seems to fit the bill moreso than the 'std' umbrella, but I wonder if there isn't something even better?

Last edited by sean on Wed Apr 26, 2006 1:30 pm; edited 1 time in total
Back to top
View user's profile Send private message
gabe



Joined: 25 Apr 2006
Posts: 4
Location: Chicago

PostPosted: Wed Apr 26, 2006 10:32 am    Post subject: Reply with quote

I'm not entirely sure what you mean by the library being 'pluggable'. Would you mind elaborating slightly? I had envisioned a system whereby the system dependant code would largely be compiled in the /os, /sys, and /lang subsections, thus releasing the average user from having to interface unnecessarily with those lower-level details. The 'extern' identifier could be moved to 'ext', of course. That would probably be even easier, as 'extension' or 'extra' is probably more idiomatic anyway. As to the 'array.d' module, it seems to me that almost everything that you have right now in /std should probably be moved to some kind of 'lang' package, as then you can subtly change the internals of those packages without affecting everything else. You can then move all of the string testing to a 'string' module in lang, etc. Also, you can then export those 'array.d' algorithms agains future implementations in packages like 'text/locale'.

One of the huge failings of java, IMHO, is that it doesn't allow for deep control, only running on top of the VM. A runtime library that embraces a substantial share of the organizational responsibility will allow anyone to program low-level if they desire, but retain the option of maintaining abstractions through a platform-neutral API.
_________________
New to the land of D, but I like the look of the place.
-Gabe
Back to top
View user's profile Send private message
gabe



Joined: 25 Apr 2006
Posts: 4
Location: Chicago

PostPosted: Wed Apr 26, 2006 12:49 pm    Post subject: Reply with quote

Also, what is the state of the source codel, i.e. what liscense is it under? Is the base fractured?
_________________
New to the land of D, but I like the look of the place.
-Gabe
Back to top
View user's profile Send private message
sean



Joined: 24 Jun 2004
Posts: 609
Location: Bay Area, CA

PostPosted: Wed Apr 26, 2006 1:37 pm    Post subject: Reply with quote

gabe wrote:
I'm not entirely sure what you mean by the library being 'pluggable'. Would you mind elaborating slightly?

Sorry, I said "module" when I meant "modular." Basically, Ares will ultimately be shipped as a small core library with a series of addons for all non-essential functionality. Therefore, if the library heirarchy is to change (and I think it should at some point) then it should be done with this in mind. The Java style layout seems like a reasonable approach at first glance, but I don't have the time to give it much thought at the moment.
Back to top
View user's profile Send private message
sean



Joined: 24 Jun 2004
Posts: 609
Location: Bay Area, CA

PostPosted: Wed Apr 26, 2006 1:46 pm    Post subject: Reply with quote

gabe wrote:
Also, what is the state of the source codel, i.e. what liscense is it under? Is the base fractured?

I really need to create a license file(s), but consider everything in Ares to be distributed under the BSD-style artistic license (though the C headers are actually public domain). However, there are a number of original authors besides myself:

* dmdrt, dmdgc: Walter Bright
* std.unicode, std.io.*: Kris (Mango)
* std.math.*: Don Clugston (Mathextra)

Some bits here and there were derived from submissions as well. Kashia supplied the Darwin-specific code, for example.
Back to top
View user's profile Send private message
FunkyM



Joined: 26 Apr 2006
Posts: 5
Location: Germany

PostPosted: Thu Apr 27, 2006 1:12 pm    Post subject: Re: library organization Reply with quote

gabe wrote:

Code:

d/
  lang
    exception
    gc
    bitarray
    boxer
    concurrency
      thread
        locks
        ...
      monitor
        ...
      ...
    reflection 
    ...



Maybe the reason why most D projects stay rather "flat" is more that people tend to come from the flat "C" grounds and big frameworks have not been created yet thus not provide a proving ground for D.

I just noticed that this kind of hierarchy is not possible in D.
Seeing some posts on the newsgroups about adding a qualified naming like "net.domain.system.drawing" have not been really taken into a real project (afaik).

D does not allow a module to be a package and does not compile.
So lets say someone attempts to do this (migrating to D you feel like this already would work):

Code:

kernel/hal.d
kernel/hal/cpu.d
kernel/hal/cpu/arm.d
kernel/hal/cpu/arm/arm7tdmi.d
kernel/hal/cpu/arm/boards.d
kernel/hal/cpu/arm/boards/netarm.d
kernel/hal/cpu/sh4.d
kernel/hal/cpu/sh4/sh7750.d
kernel/hal/cpu/sh4/boards.d
kernel/hal/cpu/sh4/boards/dreamcast.d

(Taken off: http://www.digitalmars.com/d/archives/digitalmars/D/35514.html)

This structures the code maintainable in a nice way and reflects a hierarchy that is "expected" by a developer, especially if he just learns about the language, implements a bigger framework or starts a big project (hey or simply a attempts to seperate his modules into a logical structure with multiple hierarchies).

However this fails to compile as package kernel/hal/cpu.d and module kernel/hal.d have the same name.

Alongside this issue comes that things get really bloated in a complex project as you can not split files up into smaller chunks while maintaining the same package level.

Example:
Code:

1. Intended:
system.drawing.Bitmap
system.drawing.Brush
...
This resolves to system/drawing.d containing all the classes.
Now lets say you get +20 classes with full implementaton the next logical step is to split.

2. Split:
system/drawing.d
system/drawing/bitmap.d
system/drawing/brush.d

This would be the desired layout, which however fails to compile and furthermore would introduce an undesired 2nd package level:
system.drawing.bitmap.Bitmap // expected system.drawing.Bitmap
system.drawing.brush.Brush

(Note: alias does not work to overcome System.Drawing.Bitmap.Bitmap)

I consider this the first weak point in D as far as I checked around, despite that D overall is impressive and geniously well-designed.

Not trying to make "D to Java" or anything, however this needs some rethinking in my eyes and an adaption to work, while of course maintaining the intended design goals of the module system against a namespace system.

I seen this issue be reflected in a big number of D based libraries.

Of course this can be seens as unrequired (preferably by the C enthusiasts), thus everything will be kept flat, but I don't think D should limit one into either structural approach and this issue might stick against the intentions set along with D.

I am trying to find a solution which would be logical and not break the current module system and it's intentions before I report to the mailinglists, but it seems to be not easy to find.

:)


Last edited by FunkyM on Thu Apr 27, 2006 2:25 pm; edited 1 time in total
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Thu Apr 27, 2006 2:16 pm    Post subject: Re: library organization Reply with quote

FunkyM wrote:

D does not allow a module to be a package and does not compile.
So lets say someone attempts to do this (migrating to D you feel like this already would work):


You're right, and you're not alone in how you feel about this.

However, a lot of us have adopted the convention of declaring an "all.d" where appropriate, as many of us came from Java where "*" can be used to pull in large sets of modules.

so we get this:
Code:
module system.graphics.all;

import system.graphics.Color;
import system.graphics.Canvas;
import system.graphics.Font;
import system.graphics.Polygon;
//etc...


Which covers the needed cases nicely, plus it has the effect of being more explicit when used as an import.
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
FunkyM



Joined: 26 Apr 2006
Posts: 5
Location: Germany

PostPosted: Thu Apr 27, 2006 3:02 pm    Post subject: Re: library organization Reply with quote

pragma wrote:

Code:
module system.graphics.all;

import system.graphics.Color;
import system.graphics.Canvas;
import system.graphics.Font;
import system.graphics.Polygon;
//etc...


Which covers the needed cases nicely, plus it has the effect of being more explicit when used as an import.


This indeed is an elegant "cirumvention" of the modulename!=packagename definition and enables unlimited nested package levels. However, D usually does not require any "fix/hack" in order to implement any desired programming logic; somehow only in this case. Also note that you get the following duplicated module name class layout:

Code:
system.graphics.Font.Font
system.graphics.Polygon.Polygon
// etc.


For the example it resolves to:
Code:
kernel/all.d
kernel/hal/all.d
kernel/hal/cpu/all.d
kernel/hal/cpu/arm/all.d
kernel/hal/cpu/arm/arm7tdmi.d
kernel/hal/cpu/arm/boards/all.d
kernel/hal/cpu/arm/boards/netarm.d
kernel/hal/cpu/sh4/all.d
kernel/hal/cpu/sh4/sh7750.d
kernel/hal/cpu/sh4/boards/all.d
kernel/hal/cpu/sh4/boards/dreamcast.d


However notice the additional package namespace layer and decrease in the efficent "easily guessed" practical programming D targets to provide.

Code:
kernel.hal.all.SomeEnum.UP
kernel.hal.cpu.all.OpcodesEnum.OP_ADD
kernel.hal.cpu.arm.all.ArmEnum.7T
kernel.hal.cpu.arm.arm7tdmi.arm7dmi (Class)
kernel.hal.cpu.sh4.boards.dreamcast.dreamcast (Class)

vs.

kernel.hal.SomeEnum.UP
kernel.hal.cpu.OpcodesEnum.OP_ADD
kernel.hal.cpu.arm.ArmEnum.7T
kernel.hal.cpu.arm.arm7tdmi (Class)
kernel.hal.cpu.sh4.boards.dreamcast (Class)


The module approach appears to just be the easiest for a compiler implementation (especially removes shared package level conflicts).

It's not really even about changing the language structure of D, primary a degree of added maintainability and more a "logically expected" practical solution to placing code in packages/modules and when creating APIs.

From your reply I extract that such a functionality would be a good addition in general (despite the availability of a circumvention which adds superficient package layers).
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic     Forum Index -> Ares All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group