From bradc@cray.com Thu Dec  6 14:16:38 2007
Date: Thu, 6 Dec 2007 14:16:37 -0800 (PST)
From: Brad Chamberlain <bradc@cray.com>
To: Ben Lerner <blerner@cs.washington.edu>
Subject: Re: Thoughts on using Chapel


> Your example of A[i, 1..n] is particularly tricky -- when i is holding a
> range, this has very different semantics between [] and (), even though it
> looks like there's only one range present to slice over.  If you went with
> your current suspicion and wrote it with parens today, you'd get, what, a
> diagonal slice of characters through an array of strings?

Yeah, this is all correct -- in writing that example in email, the
implication was meant to be clear that i was an integer, but you're correct
that in general it could be a range, in which case things become more
important again.


> As I understood the semantics of forall, you *might* implement the
> iterations sequentially, or in parallel; I wasn't sure which behavior
> happened when or why.  But if you were to implement forall in parallel,
> then surely the same tree-of-spawns would be used for coforall.

Typically, foralls will be implemented using something between the two
extremes of "1 thread" (for) and "1 thread per iteration" (coforall). For
domains and arrays, for example, I think of foralls as generating an
appropriate amount of parallelism for the locales onto which the object is
distributed, each of which is doing the work it owns sequentially.  In
practice, foralls on domains and arrays will be implemented in terms of the
distribution, so a user could implement it using either of the extremes,
something in the middle, or something completely different.


> But the larger point remains, if you can "compile away" coforall loops
> into even an inefficient representation using foralls and joins and
> whatever, then you've shrunk the core language.

Agreed.


> > Does anything prevent you in this case from moving the declaration of
> > the array after the sparse domain's initialization?  If not, I'd suggest
> > doing that -- declare the dense domain, the sparse domain, initialize
> > the sparse domain, now declare the sparse array.  That way, the
> > allocation happens in one fell swoop as you'd like.
> 
> I wanted all the declarations to be global, so I wouldn't have to pass
> them around in my code.

Globals are evaluated/initialized in order in Chapel, and code can exist at
the file level (it becomes initialization code for the module), so this
still ought to be workable (I have to admit that I hadn't had a chance to
connect your comments to your code, and am just starting to do so now).
Imagine:

var StartIndices: sparse subdomain(AllIndices);

// compute StartIndices += ...;

var Arr: [StartIndices] real;


But wait, you don't declare any arrays over your sparse domains!  :)



> > Or, can the sparse domain be initialized on its declaration line rather
> > than after its declaration (using an iterator, e.g.)?  This would be the
> > ideal way to do it since it would allow the sparse domain to be declared
> > const, which should result in other optimization and readability
> > benefits.
> 
> Not in my case -- the sparse domain StartIndices, for instance, is the
> location of all "<" characters in the input, and the positions just after
> all ">" characters.  That said, they are constant, right after I figure
> that information out.

This still seems like it should be doable, though.  Imagine something like:

const StartIndices: sparse subomdain(AllIndices)
                  = genStartIndices(AllIndices, str);

def genStartIndices(AllIndices, str) {
  forall z in AllIndices do {
    if str.substring[z] == '<' then {
      yield z;
    }
    else if str.substring[z] == '>' &&
            z < length(str) && str.substring[z+1] != "<" then
      yield z+1;
    }
  }
}


> Here's a wacky thought: would the following work?
> const AllIndices : [1..length(input)]
> var StartIndices : single sparse subdomain(AllIndices)  <<== note the
> single!
> var EndIndices : single sparse subdomain(AllIndices)  <<== note the
> single!
> var parsedElements : [StartIndices, EndIndices] single XmlElement  <<==
> hierarchical domains, not there yet but soon :)
> 
> /* expectation: parsedElements has not been allocated yet, since it's
> waiting for StartIndices and EndIndices to be assigned */
> 
> StartIndices = ComputeStartIndices(input); /* still waiting... */
> EndIndices = ComputeEndIndices(input);
> /* Ok, go!  Both sets of indices are available, so parsedElements can be
> allocated.  Moreover, unless it's reassigned, the domain of parsedElements
> must be constant. */

MMMmmmmaybe.  I think we've been vague/uncertain about what single should
mean in the domain context, which is the main thing that makes me nervous
about this.



> There are some BNF-style descriptions in the spec, but I'm not sure they
> match reality explicitly.  (I think the syntax for class declarations and
> deriving from base classes is missing a colon separating class D : B, for
> instance.  Also evidently semicolons are prohibited following class and
> function body definitions, but are required after variable definitions in
> class bodies.  I didn't gather that from the spec; just from the example
> programs.  I was also confused by the if-then syntax, see below.)  It's
> not so much that the spec is blatantly poorly specified, but when there
> are mismatches between the spec and reality, I don't know which is newer
> or more correct.  Having an appendix at the end containing the Bison
> grammar you actually use in the compiler (dropping the semantic actions or
> other cruft) should be an automated way to keep the spec in sync with the
> code.

OK, thanks for pointing out the cases you found that seemed wrong.  These
are bugs in the spec -- the spec's syntax is meant to describe what the
compiler currently has implemented.


> Mostly in the spec.  I found the example .chpl files very helpful.  I
> didn't get far enough into the language with this one foray to need
> anything much beyond genericStack and beer, with a few others for quick
> checks.  But in the spec, given that I wasn't sure about syntax in some
> cases, and semantics in some others (the loops, mentioned above), it would
> have helped to see a few example statements or expressions that showed
> "ok, here's what a derived class looks like; here's the options for if
> statements, etc."

OK, thanks.  We agree -- the current pass of the spec is pretty lightweight
on both examples and rationale.  We've been better at adding more as we go,
but haven't made a truly concerted effort to take a full pass through with
the intent of adding more of these.

Thanks again,
-Brad

