Note: This website is archived. For up-to-date information about D projects and development, please visit wiki.dlang.org.

Note: This project has been sleeping for I don't know how long now! The lexer and parser I wrote is now out of date .. some changes have been introduced to the language syntax and I never came around to update the project. If anyone is interested in taking over the project, you're more than welcome!



[ news | svn trunk | forums ]

Welcome to Code Analyzer

The purpose of this project is to read source code written in the D language and analyze it to extract as much information as possible. The long term goal for this project is to be useful in a full-featured IDE.

Extracting information means: being able to see it thru the eyes of the developer. To see classes, their fields and methods, figure out which ones are public, private, etc. Even more, give feed back on the fly (like Eclipse) by detecting errors and giving a descriptive explanation about its causes and how to fix it.

Another goal is to be an independent tool, not directly a part of an IDE, but usable inside it. So ideally, one should be able to use the analyzer and run it from the command line.

Short term goals include having a fully working lexer and parser; These are the first step for any serious non-trivial analysis on code. After that, we'll try to generate information necessary for code browsers, such as a list of classes/functions/templates and the location at which they're defined; and possibly parse ddoc documentation and associate them with their corresponding definitions.
This all can be usefull for static code analysis. i.e., given any source file, analyze it and generate code browing infomation.

The other problem that we aim to address is dynamic code analysis, like eclipse, given a source file that's being edited, we have to keep track of where everything is, and update the code-browsing-information dynamically as the user types his code away, while making sure that we don't lose any of the information that we had previously. It must be implemented effeciently too.
This is maybe getting too far ahead of ourselves, but it is one of the project's goals.

Project Structure

The project is divided into packages, so far, I've only managed to finish the lexical and syntax analyzers (a.k.a. lexer and parser). The lexer is in the "lexical" package, and the parser is in the "syntax" package. At the time of writing this, I have a local "semantics" package, but it's not in the svn-repo yet.
I try to write code with objects in mind, i.e. object-oriented code, but I'm not a maniac about it, I use non-object-oriented structures whenever I feel like it. The lexical analyzer works on a text-scanner object, and returns a list of token objects as the result of lexical analysis. The parser works on a token-scanner object, and returns a tree of nodes, representing the abstract syntax tree. Parsing is done with objects as well, each rule has a class that represents it. The rule-parser-classes however, are decoupled from the nodes, and effectively the nodes are just arbitrary xml-like nodes, they can carry arbitrary information, not just a parse tree, infact, the semantic analyzer deals with objects that implement the INode interface, which doesn't have anything in it that says it represents an AST node, so anything can be stored in a tree of this kind. This was done to enable saving and loading the parse tree in any format, and so the semantic analyzer is separated and decoupled from the syntactic analyzer.

Progress

The lexer is mostly done; it recognizes tokens, but at this time, it doesn't try to evaluate them, and it tolerates many kinds of errors (because when I wrote it, I didn't feel like digging too deep into the lexical grammar).
The parser is finished, but it still needs a lot of testing, I think.
Right now, I'm trying to work on the semantic analyzer, it's not a very easy task, specially since I've never wrote such an analyzer before. Any advice/references/help would be appreciated.
You can check the "news" link at the top of this page for more info on the progress.

Project Status

This project might have been abandoned since the last change to the SVN was on 2006-11-23 (r45) and a question in the forum apparently hasn't been addressed since it was posted on 2007-03-27.