Next week, eScience poster for Wednesday!

Part of what makes this really difficult is managing the session scope in the running Ruby session, while we’re running the method. This is because we don’t really want to run the method, which is normally opaque to us, instead we want to construct an appropriate scope for the method code, and walk through the method execution, tracking at each step, what the source does. Unfortunately constructing this scope is challenging, and complicated by things like yield which steps between the called method scope and the calling location scope and may be arbitrarily nested.

Ruby’s Tracing Functions

Earlier, I had looked at trying to use Ruby’s tracing functions to accomplish the whole task of determining what was happening. Tracing has it’s problems of course, especially in that a given program trace may not exercise a given section of code, because we failed to feed it the correct input information, which we can really only guess at. Similarly, the tracing information is not always rich enough to determine what exactly is going on.

A Hybrid Approach

I’m not sure why I hadn’t thought of this before, but it occurred to me yesterday, that we might be able to use a hybrid approach for this, where our tracing function is started when we go off to a method invocation, and used to give us information about what happens at this call site. This frees us from having to construct the proper scope for the method invocation and single stepping, since it allows us to use the Ruby interpreter to do this work for us. It will require a fairly fancy tracing function, because we’ll want to get back to the source code being executed, but it does free us from worrying about not tracing all code, since we will only use it for those method calls where the receiver and all arguments are already known.

Of course, even this approach is not without it’s limitations, and I’ve not thought through it enough to see how this might interact with traditional looping and branching constructs (i.e. if, for, while, etc.), but may provide enough support to solve one of the really difficult problems.

I’ve not yet had a chance to code this up (and unfortunately the end of the semester is keeping me pretty busy), but I’m hopeful that this won’t prove to be another up-hill battle!

Poster Session

Next week we have the poster session for eScience 2008, and I’ll need to get things together for a poster session.

• Finish up any work for the poster
• Generate the poster
• Get poster printed (service on campus?)

RubyWrite: Tree transformer DSL

In order to help out with a write up of the tree transformer DSL that we’ve been working on (and my advisor has been implementing), I’ve been working on implementing constant propagation and folding for a simple imperative language.

• Implement constant propagation and folding for branches (if) and looping (while)
• Implement constant propagation and folding for arrays
• Test out with a few programs

Ruby Partial Evaluation

Finally, I continue to make slow progress on the Ruby constant propagation. I’d like to get this to a point where we can make some tests of it this week, because I’d especially like to be able to put something on the poster for eScience 2008, but realistically with everything else going on, I’m not sure I’ll be able to get it to that point.

• C-base Ruby session management
  • Finish implementing proper handling of method scope for determining method safety
  • Implement session handling for swapping scopes to accommodate block/yield
• Stratego/XT Safety Decision Maker
  • Special handling for new calls
  • Implement Stratego/XT side of session handling
  • Finish handling other Ruby node-types
  • Normalize safety information
• Stratego/XT Partial Evaluator - simple evaluator using safety information

PGAS

Partitioned Global Address Space (PGAS) programming is a different programming paradigm that sort of spans the difference between shared memory programming (like Pthreads/OpenMP) and message passing programming (like MPI). It instead allows you to treat memory as if it was shared (like shared memory), but gives you control over the “affinity” of that memory for a given process. This means you are aware of what process a block of memory likely lives in, and have the ability to move it locally, perform operations on it in parallel, and in some of the PGAS languages operate with processes as “teams” on the data.

All of this is pretty cool, but two things really stuck out. The first, is that the most mature of the PGAS languages Unified Parallel C (UPC) is implemented as a handful of extensions on top of ISO C99, and hence it might be possible to roll a version of Ruby that could take advantage of concepts from PGAS by compiling Ruby using UPC, and writing a UPC extension to all for experimenting with Ruby PGAS.

The other thing I thought was really interesting about the PGAS languages, are that two of the languages (X10 from IBM and Chapel from Cray) are targeted at high productivity and both provide some concept of object-oriented programming along with closures, making them a great deal more flexible then the traditional Fortran or C/C++ with MPI.

Partial Evaluator

My number one priority this week is getting the basics of the safety checking working for methods, and finishing the parts of the partial evaluator that will let me walk a full Ruby AST. The list of the work to be done, more or less in order, though some of the more advanced handling may be done after a first pass of the partial evaluator is finished for simpler cases.

• Safety check
  • Safety checking of initialize methods for newly created objects
  • Safety checking of non-initialize methods for existing objects and their parameters
  • “Kill list” for variables that have not yet gone out of scope in the Ruby session, but can no longer be used because they may have been changed by an unsafe expression
  • Finish tree-walking for other node types, including handling of branching and looping
  • Special handling for non-new initializers
  • Special handling for require and load and code that calls it
• Simple partial evaluator
  • Evaluate safe statements once (and only once)
  • Output evaluated versions of anything that results in a literal
  • Dead code elimination step, after partial evaluation

While I probably will not be able to get all of this done, I’d at least like to get the basic safety parts working for full Ruby ASTs under the assumption that require and load will not be called.

Fellowship Applications

I did finish up my fellowship applications, with the exception of getting information to my reviewers which I should get done tonight.

Ruby Partial Evaluator

I haven’t really made much progress on this, though I did get a chance to start working on this again a little today. In reflecting on it a little bit, I had mentioned treating calls to new special in two ways:

1. We need to check the initialize function for the implementation, rather then new
2. We can allow code in initialize (or functions it calls) to set instance variables on the newly created object

The second condition here though, I think could be broader. Since any method call can potentially add a new instance variable, I think it may be helpful to maintain a hash of the defined instance variables for a given instance of a class, and always allow the first assign to go without counting it as a modification (since it is really just an extension of the object, rather then a modification of its state). I think this will give us more precise information, though it may make it a little more difficult to be precise about this, since it is not uncommon to have initializer methods other then new or methods that add new instance variables the first time they’re called.

Fellowship Applications

• Send references requests off, with a packet of the information I had sent off to NSF
• NPSC Fellowship Application
  • Finish up these essays, largely rewriting and polishing at this point
  • Complete their online forms
  • Put together an equivalent packet for my reference writers for this stuff as well

Ruby Partial Evaluation

• Finish code that extracts properties of a method call (specifically constant, global, class, and instance variables modified or referenced, methods called, and which methods are called on objects passed in as arguments)
• Use properties of method call for special casing new method call (i.e. have it look at initialize
• Begin working on algorithm for determining which side-effects we can partial evaluate, and which will need to wait for runtime.
• Create simple partial evaluator that uses the safety information from the offline analysis to create the partially evaluated code.

Sad Mac News

Also, in sad mac news, my MacBook Pro started to randomly kernel panic on me over the weekend and by Sunday afternoon, was pretty consistent at dieing after 10-15 minutes of up time (at this point it is even less!) So I’m a little off schedule from this weekend. On the plus side, I didn’t loose anything, as I’ve pulled stuff I’ve been working on recently onto my home Linux server, and I’ve got AppleCare on the laptop, so hopefully they’ll be able to fix me up. Still, sad not to have a working laptop!

Fellowships

It was a little down to the wire, but I managed to get my NSF Graduate Student Fellowship application in. The NPSC folks, extended the deadline by a week, so I’ll be finishing that up over the weekend.

Ruby Partial Evaluation

I’ve started working again on the Ruby partial evaluator, specifically focusing on the safety analysis piece of this code. I’ve gotten back to roughly the point where I was before, starting to look at extracting the properties of a function call at a given call site in order to figure out if it is safe or not.

Fellowship Applications

• Write application essays
• Finish filling out application sheets (by Wednesday)

Partial Evaluator

• Start work on iterative analysis to determine which expressions are safe to evaluate and which are not
  • Initial pass: data-flow analysis to setup known variables/objects and initial method call safety list
  • Iterative pass: using discovered data, extend safety to new expressions, and re-evaluate

At a guess it will be at least next week before I get a chance to begin implementing the partial evaluator that uses this data, even if the safety analysis tool goes fairly smoothly, but once the safety stuff is working, it should make the partial evaluation run relatively quickly.

Fellowship Applications

Unfortunately, I didn’t get a lot done this week. I realized on Tuesday night, that the fellowships I’m applying for are due on the 5th not the 9th, so I’ve been mostly focused on getting these applications ready, though I’ve still got quite a bit to do on that front before I’m finished.

Ruby Partial Evaluator

While I didn’t get a chance to look much at the coding aspects of this, I did have a very good conversation about how to approach this with my advisor, and once my fellowship applications are ready, I’m looking forward to getting back to this.

The basic idea is pretty straight forward, we’re going to treat determining the safety of running expressions in Ruby as an iterative data-flow analysis problem.

• Initial pass
• A Ruby session, with fresh scope will be started
• Assignments of literals will be evaluated in the session
• Assignments of newly created objects will be evaluated, if no global or class variables are accessed, or functions called (other then known safe functions)
• The require and load functions (and methods that call them) will need special treatment to properly handle library inclusion.
• All other method calls will be evaluated into one of three categories
• Safe: No global, class, or instance variables accessed, and only known safe functions are called
• Unsafe: A known unsafe method is called
• Unresolved: Global, class, or instance variables are accessed, or unresolved functions are called, then an annotation is added to list the global, class, or instance variables set or read, and the methods called by each, along with what arguments are updated (if known) or what methods are called on these
• Iterative Pass
• The current Ruby session scope is thrown away, and a fresh one established
• The same assignments are originally computed are performed, along with any new method calls determined safe
• Once a method is determined to only be unsafe because of modifying variable state, we look to prove that these modifications can be propagated to all method calls referencing a given global, class, or instance variable, and can all be executed.
• Now that more methods can be executed, more information is known about the program, and it can be refined
• Once no more refinements can be made (i.e. a fixed-point is reached) we stop, the result is the original AST annotated with what is and is not safe

Once the basics are implemented, we’re hoping to extend this method to handle things like libraries and partial evaluation within methods, where properties (such as core library method definitions) hold.