Chapel

Chapel is a new parallel programming language being developed by Cray Inc. as part of the DARPA-led High Productivity Computing Systems program (HPCS). Chapel is designed to improve the productivity of high-end computer users while also serving as a portable parallel programming model that can be used on commodity clusters or desktop multicore systems. Chapel strives to vastly improve the programmability of large-scale parallel computers while matching or beating the performance and portability of current programming models like MPI.

The Chapel app supports a multithreaded execution model via high-level abstractions for data parallelism, task parallelism, concurrency, and nested parallelism. Chapel's locale type enables users to specify and reason about the placement of data and tasks on a target architecture in order to tune for locality. Chapel supports global-view data aggregates with user-defined implementations, permitting operations on distributed data structures to be expressed in a natural manner. In contrast to many previous higher-level parallel languages, Chapel is designed around a multiresolution philosophy, permitting users to initially write very abstract code and then incrementally add more detail until they are as close to the machine as their needs require. Chapel supports code reuse and rapid prototyping via object-oriented design, type inference, and features for generic programming.

Chapel was designed from first principles rather than by extending an existing language. It is an imperative block-structured language, designed to be easy to learn for users of C, C++, Fortran, Java, Perl, Matlab, and other popular languages. While Chapel builds on concepts and syntax from many previous languages, its parallel features are most directly influenced by ZPL, High-Performance Fortran (HPF), and the Cray MTA™/Cray XMT™ extensions to C and Fortran.

What is new in this release:

• Highlights (see below for details)
• split 'def' keyword into 'proc' (for procedures) and 'iter' (for iterators)
• (see '$CHPL_HOME/util/convert-defs --help' for help updating existing codes)
• added 'Replicated' distribution and improved 'Block-Cyclic' distribution
• (see "Standard Distributions" in spec and examples/primers/distributions.chpl)
• added single-locale task layers for Nanos++ (BSC) and Qthreads (Sandia)
• (see doc/README.tasks)
• major reworking of range semantics and implementation
• (see "Semantic Changes / Changes to Chapel Language" below for detail)
• initial support for volatile types: bool, int, uint, real, imag
• (see doc/technotes/README.volatile)
• support for extern consts and classes (pointers-to-structs)
• (see doc/technotes/README.extern)
• improved support for data parallelism on the Cray XMT
• extensive improvements to the sample codes in the examples/ directory
• (see 'Example Codes' below and examples/README)
• added most code examples from the spec to a new examples/ directory
• (see examples/spec/*)
• added a technical note describing the user-defined domain map interface
• (see doc/technotes/README.dsi)
• extensive updates to the language specification
• (see 'Documentation' below)
• several improvements to the runtime tasking and communications layers
• (see "Runtime Library Changes" below)
• Syntactic/Naming Changes
• split 'def' keyword into 'proc' (for procedures) and 'iter' (for iterators)
• (see '$CHPL_HOME/util/convert-defs --help' for help updating existing codes)
• renamed 'arithmetic' domains/arrays to 'rectangular' domains/arrays
• added support for floating point literals of the form '1.e...'
• (previously, only '1e...' and '1.0e...' were supported)
• Semantic Changes / Changes to Chapel Language
• several improvements to ranges (see 'Ranges' chapter in the language spec):
• added a concept of alignment to indicate an alignment modulo |stride|
• added an 'align' operator and query to ranges to specify/query alignment
• (e.g., '1..10 by 3 align 2' => '2, 5, 8' since 2 == 5 == 8, mod 3)
• defined == on ranges in terms of the integer sequences they generate
• defined ident(r1, r2) to say whether r1 and r2 are semantically identical
• defined '# k' for k < 0 as meaning "count from the end of the range"
• added 'first', 'last', 'alignedLow, 'alignedHigh' methods to ranges
• (e.g., '1..10 by -2' => first==10, last==2, alignedLow=2, alignedHigh=10)
• added support for a variety of new range query functions
• (e.g., alignLow(), hasFirst(), hasLowBound(), isAligned(), boundsCheck()...)
• added support for volatile bool, int, uint, real, and imag types
• (see doc/technotes/README.volatile)
• added an end-of-file check to the file type
• (e.g., 'while !infile.eof ...')
• removed support for treating 'return' in iterator as a 'yield'
• (e.g., change 'iter foo() { return x; }' to 'iter foo() { yield x; return; }')
• added support to referencing modules without first 'use'-ing them
• (e.g., module M1 { var x...} module M2 { ...M1.x... } is now legal)
• added a callStackSize query to the locale type
• (e.g., 'here.callStackSize' returns the call stack size on the current locale)
• removed the previously-required semicolon from the end of enum declarations
• (e.g., 'enum colors {red, blue, green}' is now legal; previously needed ';')
• added support for a single trailing comma at the end of enum lists
• (e.g., 'enum colors {red, blue, green, }' is now legal)
• removed support for +/- operators on rectangular domains; use translate()
• Newly Implemented Features
• implemented ==/!= for rectangular, associative, and sparse domains
• Standard Distributions
• added a first-draft 'Replicated' distribution for storing data redundantly
• (see examples/primers/distributions.chpl)
• improved 'Block-Cyclic', sufficient to correctly implement HPCC PTRANS and HPL
• (see examples/hpcc/ptrans.chpl, hpl.chpl; examples/primers/distributions.chpl)
• removed redundancy in specification of leader iterators between domains/arrays
• Standard Modules
• added INFINITY and NAN constants to the Math.chpl module
• (see 'Math' under "Standard Modules" in language spec)
• added isinf() and isfinite() test routines to the Math.chpl module
• (see 'Math' under "Standard Modules" in language spec)
• added a half dozen more routines to the GMP.chpl module
• Documentation
• added a new technical note describing the user-defined domain map interface
• (see doc/technotes/README.dsi)
• made many improvements to the language specification
• major revisions to the Domains chapter to improve clarity/accuracy
• improved and unified structure of Records and Classes chapters
• major revisions to the Domain Maps chapter to improve clarity/accuracy
• named example codes to provide cross-reference to examples/spec/ programs
• clarified the legality of label/break/continue in parallel loops
• updated the descriptions of read[ln]/write[ln]
• fixed some problems in file fields ordering
• reformatted the specification as a 'book' in LaTeX
• various minor fixes of typos/mistakes
• improved definitions/descriptions of several terms
• added documentation of locale.name
• and many other changes...
• updated quick reference document to include def->proc/iter change, align op.
• rewrote/reorganized README.extern to clarify explanations and bring up-to-date
• added a README describing the 'local' statement (doc/technotes/README.local)
• added a README describing volatile types (doc/technotes/README.volatile)
• updated README.comm-diagnostics to mention fast fork statistics
• updated various README files
• Example Codes
• placed most code examples from the language spec. to a new spec/ subdirectory
• improved the primers/ directory:
• added a new primer for using procedures (primers/procedures.chpl)
• added a basic primer for domains (primers/domains.chpl)
• renamed the locales primer to locales.chpl and improved it
• improved the distributions primer and added Block-Cyclic and Replicated uses
• updated the range primer (primers/range.chpl)
• improved the iterators primer (primers/iterators.chpl)
• improved the reductions primer (primers/reductions.chpl)
• added getCurrentTime() to the timers primer (primers/timers.chpl)
• added descriptions of type aliases and config types to variables primer
• (see primers/variables.chpl)
• improved hpcc/ benchmarks directory
• added first version of HPCC PTRANS to hpcc/ directory (ptrans.chpl)
• added cleaner version of HPCC HPL to hpcc/ directory (hpl.chpl)
• added a variants/ subdirectory to examples/hpcc to exhibit other approaches
• added stream-promoted.chpl to show use of promotion in global STREAM Triad
• added ra-cleanloop.chpl to show cleaner (but currently slower) RA kernel
• improved top-level examples:
• renamed existing "hello, world" examples to suggest a logical reading order
• (see examples/hello*.chpl)
• added data parallel and distributed data parallel hello, world examples
• (see examples/hello3-datapar.chpl, examples/hello4-datapar-dist.chpl)
• added comments to the various hello, world examples (examples/hello*.chpl)
• updated all examples to current language semantics
• use 'proc'/'iter' rather than 'def'
• update to reflect new range semantics
• Platform-specific Notes
• for Cray XT/Cray XE, improved the aprun and pbs-aprun launchers
• (see "Launcher-specific notes" below)
• for Cray XMT, added support for parallel loops over ranges
• for Cray XMT, added generation of noalias pragmas to forall loop variables
• added support for BSC's MareNostrum including docs and a special launcher
• (see doc/platforms/README.marenostrum)
• Launcher-specific notes
• added launcher-specific options to --help capability
• improved propagation of exit status codes through launchers
• changed launchers to support exec-based (rather than system-based) commands
• made launchers use 'unlink' rather than system('rm') to remove temporary files
• added a launcher for BSC's MareNostrum
• reduced number of temporary files used to capture output of 'system' commands
• aprun launcher improvements:
• added a --cc flag to specify CPU assignment w/in a node
• added a -q flag to run the launcher in quiet mode
• added debug capability via CHPL_LAUNCHER_DEBUG to keep tmp files around
• pbs-aprun launcher improvements:
• added a --cc flag to specify CPU assignment w/in a node
• added debug capability via CHPL_LAUNCHER_DEBUG to keep tmp files around
• cleaned up output to get rid of some extraneous printing
• made launcher support our testing system
• added --walltime and --queue flags to launcher as alternatives to env. vars.
• Compiler Changes
• made the compiler print 'true'/'false' out in type signatures rather than 1/0
• added config param 'noRefCount' to disable reference counting (may leak memory)
• Compiler Flags
• added a --print-callstack-on-error flag to show what led to a bad call
• added a --no-codegen flag to skip the code-generation and linking phase
• Interoperability Changes
• added support for extern C "classes" (pointer-to-struct types)
• added support for extern consts (see README.extern)
• Error Message Improvements
• added a warning orphaned 'use' statements outside of declared modules
• (e.g., 'use M1; module M2 { }' should typically be 'module M2 { use M1; }')
• made parse-time error message use the standard 'filename:lineno:' format
• generated an error message for referring to a class name in a primary method
• (e.g., 'class C { def C.m() {...} }' now generates an error message)
• Bug Fixes / New Semantic Checks (for old semantics)
• prevented config types from being set using execution time command-line flags
• fixed the compiler's ability to handle unions with no fields
• fixed the minloc/maxloc reductions to handle empty ranges/subranges of values
• fixed a bug in which the 'path' argument to file constructors was ignored
• made empty unions work correctly
• fixed an overflow issue in chunking up ranges for distribution/parallelization
• fixed a bug involving support for empty extern record types
• fixed a bug involving extern functions that return records
• fixed a bug related to copying external records
• fixed implementation of +/- operators on associative/sparse domains
• fixed a bug relating to filename-based module names containing multiple '.'s
• added error messages for expand/exterior/interior on irregular domains
• added an error for constructors/destructors with a specified return type
• fixed a pair of bugs relating to the 'fast on' optimization
• added a check against constructing classes without specifying generic fields
• Packaging Changes
• added syntax highlighter code for GNU source-highlight
• (see etc/source-highlight/README)
• improved emacs coloring support for version 22.x
• (see etc/emacs/README)
• updates to vim syntax coloring to reflect new 'proc'/'iter' keywords
• (see etc/vim/README)
• added a script for converting existing codes' use of 'def' to 'iter'/'proc'
• (see $CHPL_HOME/util/convert-defs)
• removed multirealm examples & documentation for the time being
• Third-Party Software Changes
• updated GASNet to version 1.16.1
• added copies of BSC's Nanos++ and Sandia's Qthreads
• (see third-party/README)
• made a post-install step for GASNet that changes absolute paths to relative
• removed PVM for the time being due to instability
• Runtime Library Changes
• added single-locale task layers for Nanos++ (BSC) and Qthreads (Sandia)
• several improvements to the runtime tasking layer:
• added a new interface for having the tasking layer call chpl_main()
• changed tasking init function to take maxThreadsPerLocale/callStackSize args
• added a task yield capability to the tasking layer interface
• improved runtime code to avoid using sync vars before initializing tasking
• removed macro-ized form of tasking interface; using standard names instead
• unified naming of all task layer routines to chpl_sync_*() and chpl_task_*()
• removed requirement to implement the single variable interface
• moved termination of threads from fifo tasking layer to pthreads threading
• simplified implementation of sync variables in the pthread threading layer
• moved responsibility for thread counts from fifo tasking to pthread threading
• several improvements to the runtime communication layer:
• restructured the communication interface to support length/type arguments
• added an interface for non-blocking get operations to the comm. interface
• unified all remote get operations to use CHPL_COMM_GET() macro
• refactored/renamed the communication interface headers
• Testing System
• vastly improved the performance testing and graphing capabilities
• (see comments at the front of start_test for documentation)
• improved the testing system's support for cleaning up after itself
• added ability to avoid using any indirection of stdin
• added an option to provide a system-wide prediff option
• added ability to use a launcher's native timeout feature rather than Python's
• improved how the testing system kills a test that times out
• fixed a bug in which .preexec files only worked if '.' was in your path
• Internal
• added a --break-on-id compiler flag for developers to locate AST node creates
• improved organization of util/ directory contents
• changed the way version numbers are computed/displayed for SVN-based users
• added support for task-/thread-specific module code
• added a capability to print the AST counts allocated between each pass
• added a new gdb alias to compiler --gdb: 'loc' prints an AST node's location
• greatly improved implementation of pragmas/flags to avoid error cases
• removed several instances of unnecessary string comparisons in the compiler
• removed error-prone use of CHPL_TASKS and CHPL_THREADS as preprocessor symbols
• added bestPractices documentation for developers (not included in release)
• changed internal munging of '=' and '==' to 'ASSIGN' and 'EQUALS' respectively
• added a --print-id-on-error developer flag to print AST ID on errors
• improved the strictness of C++ prototype checking
• added external contributor agreements to the SVN tree (not in release)
• added a notion of internal types used by the compiler but not the user