FFA split into two packages

Besides providing `foreign friendly' arrays, my FFA package contained a lot of functions that implement commonly used array operations, including elementwise algebra operations, array minima and maxima, etc. Because these do not depend on CFFI, I decided I would put them into a separate package, named ARRAY-OPERATIONS, which is also installable with ASDF-Install. It contains a reimplementation of array-range, and also a macro for vectorizing operations, called vectorize:

 (let ((a #(1d0 2d0 3d0))
       (b #(4d0 5d0 6d0))
       (c 0.25d0))
   (vectorize (a b)
              (+ a b c 0.25)))  ; => #(5.5d0 7.5d0 9.5d0)

(vectorize (&rest vectors) expression) sets up symbol macros so that vector names in vectors refer to the appropriate element, then evaluates expression repeatedly and returns the results as a vector. It also has two keyword arguments: result-element-type (which default to 'double-float) determines the element type of this vector, while into places the results in one of the vectors instead (of course the two arguments are mutually exclusive).

 (let ((a (vector 1 2 3)))
   (vectorize (a) (+ 1 (* a 2)) :into a)
   a)  ; => #(3 5 7)

Functions in the new package still strive to create arrays that are foreign-friendly.

announcement: ffa

While CFFI is very useful API for calling C functions, using foreign functions that expect pointers to arrays has always been something that is tricky to do in Common Lisp. The two basic approaches are

1. allocating a chunk of foreign memory and accessing it from Lisp as needed (as implemented in the fnv package by rif)
2. using native CL arrays and either copying them to/from foreign memory on demand, or taking advantage of implementation-specific features (eg pinning arrays) for direct access where possible

The first approach has some advantages, but I find that I am more comfortable with the second one, even if it leads to some loss in speed when the arrays are large and the implementation can't provide direct access for foreign functions. My ffa (Foreign Friendly Arrays) package provides some convenience functions and macros that take advantage of implementation-specific features when available. The most important features are the function make-ffa and the macro with-pointer-to-array. The function make-ffa is similar to make-array, except that it recognizes some CFFI types (eg :double, :uint32) and when asked to allocate a multidimensional array, it will do so by displacing a simple array which it creates first. This allows direct access in some implementations, especially SBCL, while providing multidimensional arrays at the same time. The macro (with-pointer-to-array (array pointer cffi-type length direction) body) makes sure that the contents of array will be mapped to a memory area designated by pointer during the execution of body. cffi-type determines the foreign element type, direction tells the macro whether to copy the array to and/or from the memory area (so, for example, if you foreign function needs data but does not modify it, you don't need to copy it back), and by providing length, you ensure that the array contains as many elements as your foreign function needs. When possible, no copying/conversion will take place (eg a double-float array in SBCL with cffi-type :double), but the macro will take care of it automatically when needed (issuing some efficiency warnings when practical). Displacing all arrays from a one-dimensional simple array also allows a few convenience functions, please check the code or the tutorial (provided in the package) for details. Currently, only SBCL-specific code is provided, for all other implementations, the macro defaults to copying/conversion. Contributions for other implementations and suggestions for improvements are welcome. I would like to thank all those who commented on an earlier version of ffa on lisp-matrix-devel.