Caml Weekly News Previous week Up Next week

Hello

Here is the latest Caml Weekly News, for the week of January 09 to 16, 2007.

  1. compiling wxocaml (and wxhaskell)
  2. color on linux terminal
  3. Learn OCaml or the bunny gets it
  4. glome-0.2 (ocaml-based raytracer)
  5. va_arg values

compiling wxocaml (and wxhaskell)

Archive: http://groups.google.com/group/fa.caml/browse_frm/thread/25c4834885e75e78/6f6729a1e54a5124#6f6729a1e54a5124

Dom. Orchard added to this old topic:
I have been trying to do this too and have finally managed to get wxHaskell 
0.9.4 working with wxWidgets 2.8.0. 

My guide + code to do it can be found here: 
http://riftor.g615.co.uk/content.php?view=53&type=2 

I hope that it works for you too, it should do! Let me know if you have any 
problems, 

Dominic Orchard
			

color on linux terminal

Archive: http://groups.google.com/group/fa.caml/browse_frm/thread/b99a2a27411acf50/e35861a581612fe9#e35861a581612fe9

Markus Weihs asked:
to get coloured output on a linux terminal, you can do something like 

    echo -e "\033[31m This is now red" 

How can I do this with OCaml? The following doesn't work 

    print_string "\033[31m blabla"
			
Daniel de Rauglaudre answered:
print_string "\027[31m blabla" 
Character codes are in decimal in OCaml, not in octal.
			
Goulagman said and Philippe Wang added:
> It's because, for Ocaml,   the escape sequence \0xx matches the   
> ASCII character xx in decimal, not in octal. 
> print_string "\027[31m blabla" works fine 

Well, what you said is true, still it's not meant to be explained   
like this ! (well I guess and hope so) 
The backslash in a character or string sequence introduces a decimal   
number between 0 and 255 with *exactly* 3 digits. 

You can also write special characters in hexadecimal : 

"\xFF"  = "\255" 

;-)
			
Christophe TROESTLER also answered the OP:
You can also use ANSITerminal 
http://math.umh.ac.be/an/software.php#x4-90008.
			

Learn OCaml or the bunny gets it

Archive: http://groups.google.com/group/fa.caml/browse_frm/thread/4c04a121b96a1326/dab496cd9ee244d6#dab496cd9ee244d6

Jon Harrop announced:
Having read a thread about a real-time ray tracer by Jacco Bikker rendering 
the Stanford bunny at 7fps, I decided to try rendering the bunny from OCaml. 

Chucking it into an OpenGL display list gets me about 450fps on my machine 
(2.2GHz AMD64, GeForce 7900GT) with 16x FSAA for only 85 lines of code: 

  http://www.ffconsultancy.com/free/bunny/ 

I'm working on shadows...
			

glome-0.2 (ocaml-based raytracer)

Archive: http://groups.google.com/group/fa.caml/browse_frm/thread/899842ff3c645c8d/74c5d0565cd66ce0#74c5d0565cd66ce0

Jim Snow announced:
I've been working on a raytracer for awhile, and recently decided to 
remove a lot of experimental code that doesn't work well anyways and 
release the rest under the gpl version 2.  Currently, glome renders some 
of the scenes from the standard procedural database 
(http://www.acm.org/tog/resources/SPD/).  I thought that, aside from the 
practical utility of generating pretty pictures, some people on this 
list might be interested in using it to benchmark the quality of code 
generated by various versions of the ocaml compiler. 
Supported primitives are spheres and triangles.  It uses a kd-tree as an 
acceleration structure.  There is limited joystick support (moving works 
fine, but turning can have unexpected results) for those patient enough 
to tolerate the low framerates. 

I use lablgl for screen output, but there aren't any other libraries 
required outside of the standard ocaml distribution. 

I'm not a very experienced ocaml programmer, so I'm sure there are some 
things I'm doing inefficiently just because I don't know better.  I 
welcome any suggestions that would make my code faster, or reduce the 
memory footprint of my scene representation. 

There is a discussion thread about glome over at ompf.org: 
http://ompf.org/forum/viewtopic.php?t=336 

Source code download is here: http://syn.cs.pdx.edu/~jsnow/glome/
			
Jon Harrop said and Jim Snow answered:
> > I've been working on a raytracer for awhile, and recently decided to 
> > remove a lot of experimental code that doesn't work well anyways and 
> > release the rest under the gpl version 2.  Currently, glome renders some 
> > of the scenes from the standard procedural database 
> > (http://www.acm.org/tog/resources/SPD/).  I thought that, aside from the 
> > practical utility of generating pretty pictures, some people on this 
> > list might be interested in using it to benchmark the quality of code 
> > generated by various versions of the ocaml compiler. 

> I have altered the code to be more idiomatic OCaml, although it is still very 
> not-OCaml. I've removed OOP from the hot path and virtual function dispatch 
> has been replaced with pattern matches. 

>   http://www.ffconsultancy.com/temp/glome.tar.bz2 

> The code is now 1390LOC instead of 1746 (20% shorter). Performance is also 
> better. Building the Kd-tree is down from 7.0s to 6.3s. 


Sorry I'm a bit slow about replying; I was off trying to implement an 
nlogn kd-tree compiler.  Your version seems to have sped up the 
raytracing by about 10%.  However, I think I am going to stick with my 
approach for the time being for the sake of maintainability; I don't 
think putting all the ray-intersection code together in one 
mutually-recursive is going to make the program easy to modify in the 
future.  I am tempted though.  I might also give recursive modules a try. 
(For those just joining us, my dilemma is thus: my raytracer defines 
ray-intersection tests for a number of types of geometry.  Aside from my 
conventional primitives, triangles and spheres, I also have a number of 
more abstract primitives like groups (a container for primitives, so I 
can treat, say, a collection of triangles as if it were one triangle) 
and kdtrees (semantically similar to a group, but with an axis-aligned 
binary space partitioning scheme).  In order for this latter type to 
work correctly, they need to have a ray-intersection function that calls 
the ray-intersection functions of their contained objects.  Contained 
objects may also be groups or kdtrees, hence the necessity of mutual 
recursion.  Due to the lack of mutual recursion across source files, I 
had resorted to using objects; all primitives inherit from a base type 
that supports a ray-intersection test.  Unfortunately, this incurs 
noticeable overhead.  Jon Harrop's solution was to write one big 
recursive ray-intersection test that pattern matches on the type of 
supplied primitve, and evaluates the proper test.) 

> I have many suggestions for what to do next: 
> 1. Use records instead of float arrays: stronger type inference, more 
> concise, purely functional. 


I did try this after looking at your ray tracer; however, this did not 
significantly affect performance, except in cases where I needed to 
access vectors like an array (with an integer index), and none of the 
tricks I could think of to do that were as fast as plain array access.   
This created a bottleneck in my kd-tree traversal code (where 
high-performance ray tracers tend to a significant portion, if not most, 
of their time). 
See http://ompf.org/forum/viewtopic.php?p=2709&highlight=#2709 

> 2. Get rid of almost all mutation. The core ray tracer has no reason to use 
> mutation and all those refs and assignments are confusing and probably slow. 

If you mean the place where I pass a "traceresult" record into each 
rayintersection test, I agree that that is definitely ugly, but at the 
time it gained me a noticeable performance increase (I think it was 
around 5% or so.).  However, that was under a different workload; that 
was before I implemented the kdtree and I was doing many more ray-object 
intersections.  I was also optimizing for a different set of 
ray-intersection tests that returned more information, and I was keeping 
the results around much longer (I couldn't throw them away until the 
whole image was done rendering).  I didn't want them to sit around so 
long they get promoted to the major heap, only to get garbage collected 
much later.  It's possible those concerns aren't valid for the way the 
raytracer currently works.  I might try reverting that optimization one 
of these days and seeing what happens. 
If assignment to mutable fields bothers you, I suggest you avert your 
eyes from the clr.ml file.  I don't really have a good excuse for that 
code, other than it seemed like a good idea at the time and I haven't 
gotten around to re-writing it. 

> 3. Restructure the program again: put independent definitions related to 
> triangles in Triangle, put related definitions like the intersection routine 
> in Intersect. 
> Primarily, the program is far too verbose and convoluted. As an algorithm,
> ray tracing is very functional in nature. I think the functionality provided 
> by this program could be achieved in half as many lines of code. It could  
> also be a lot faster. 


You're right that the program could be cleaner and much shorter, but I'm 
relatively new to functional programming, and I haven't figured out all 
the shortcuts.  When I'm feeling lazy and I see something that seems 
like it should be a loop I'll usually use a loop, whereas I could have 
used recursion and saved myself some typing.  I like that ocaml doesn't 
force you to program in a particular way. 
I also don't think "lines of code" is always a good way of measuring 
code quality.  Oop, for instance, adds a lot of cruft (which is one 
reason I dislike java; I don't like being forced to do all that typing), 
but I used it because it gave me mutual recursion without having to 
stick all my mutually recursive functions together in one file, and 
therefore I can group my code into smaller, more manageable units.  (I 
won't dispute that your version is faster.) 

> > Supported primitives are spheres and triangles.  It uses a kd-tree as an 
> > acceleration structure.  There is limited joystick support (moving works 
> > fine, but turning can have unexpected results) for those patient enough 
> > to tolerate the low framerates. 
> > I use lablgl for screen output, but there aren't any other libraries 
> > required outside of the standard ocaml distribution. 

> Rather than rendering dots, you could generate a polygon mesh. To make 
> things more interesting, you could include the depth value in the mesh, so 
> when you rotate the scene it gets distorted by OpenGL without needing to ray 
> trace anything. 

Hm, I'll bet you'd like to know what the 2/3 of the code I didn't 
publicly release does :) 
(It doesn't do quite what you suggest, but I do draw the final image to 
the screen as an adaptive triangle mesh (using the ROAM algorithm).) 
Your bunny renderer looks interesting.  It's been on my to-do list to 
contrive some way to load more interesting datasets than the standard 
procedural database.
			
brogoff then added:
I haven't coded a ray tracer in a long time, and the one I did was for a 
college class, but my recollection is that even in C (the implementation 
language I used) the design used classes/objects for the primitives, so 
that one could add new primitives and the only piece of code that would 
need modification would be the interpreter of scene description files. 
I think using classes for that is the right approach. I'm sure you could 
do it in a more coreish ML fashion without even using recursive modules, 
say by emulating the OO in C approach with ML records of functions, but 
it won't be any faster and will be uglier, since the class system 
provides a kind of generalized polymorphic record. 

This is a nice example for discussing the merits of OO features, and less 
complex than the expression problem. A competitive "non-OO" approach 
should provide easy extensibility along the same axes as the OO one. 
I admit I don't see the need for cross file recursion here.
			

va_arg values

Archive: http://groups.google.com/group/fa.caml/browse_frm/thread/9663c0be0dfa2a6d/d00690ea470fc554#d00690ea470fc554

Bob Matcuk asked:
I've been wondering what is the best way to write a C function that 
takes a variable number of value arguments. Lets say, for example, 
that I was writing a function that took an object, a string 
(specifying a method on the object), and a variable number of 
arguments to pass to the method. The function would then construct an 
array (with the object being the first element) from these arguments 
and pass it along to caml_callbackN. This function, of course, would 
only ever be called by other C functions. 
The thing that I guess I'm caught up on is the fact that I cannot 
directly apply CAMLparam to these variable arguments. Some calling 
conventions place all arguments on the stack, in which case CAMLparamN 
could be used (as long as you knew whether the stack was building up 
or down). However, some do not do this (the AMD64 calling convention, 
for example, puts the first 6 arguments in registers, the rest on the 
stack). 

I guess the real question is: is it even necessary to worry about this? 
The function, as I said, will only ever be called from other C 
functions (who have already designated these values as being 
params/local to themselves, assuming they are written correctly). I 
seem to recall reading somewhere that if you write a function that 
will only ever be called from other C functions that have already 
registered the values (via CAMLparam/CAMLlocal), then it's 
unnecessary to do it again. The function doesn't allocate any new 
values either, so it shouldn't trip the GC anyway, right? The function 
should, therefore, be something like this: 

value func(value obj, char *method, int n, ...) 
{ 
  va_list ap; 
  int i; 
  value *args = calloc(n + 1, sizeof(value)); 
  value r; 

  args[0] = obj; 
  va_start(ap, n); 
  for (i = 0; i < n; i++) 
    args[i + 1] = va_arg(ap, value); 
  va_end(ap); 

  r = caml_callbackN(caml_get_public_method(obj, hash_variant(method)), 
                     n + 1, args); 
  free(args); 
  return r; 

} 

Should probably check calloc for success and maybe throw an exception 
if it failed... Which actually brings me to another quick question: if 
I throw an exception, say caml_failwith("..."), is it necessary to 
still call CAMLreturn after it? Or will the exception cause the 
function to exit? 
Is it an invalid assumption that it is unnecessary to bother with the 
CAMLparam/CAMLlocal stuff since there's nothing to trip the GC? If so, 
what is the best way to handle all the CAMLparam/CAMLlocal stuff? For 
example, CAMLlocalN(args, n + 1) is invalid because C does not allow 
you to declare dynamic arrays. Looping over the arguments with 
va_start/for loop/va_arg and calling CAMLparam on them is also invalid 
because it would be declaring the caml__root_##x variable within the for 
loop. I have typed up some code that should work if it is necessary, 
but it's messy and if CAMLparam is ever changed, it's likely I'll need 
to change my code too. I want to avoid that. 

One last quick question: is the line "args[i + 1] = va_arg(ap, value);" 
above legal? args[] is an array of value and va_arg(ap, value) will 
return a value. So, essentially, it's the same thing as the assignment 
in the following example: 

value func(value v1) 
{ 
  value v2 = v1; 
  ... 

} 

I know values are just pointers so it is syntactically correct, but 
what I'm asking is: is it safe to do? Should I be using some function 
instead to create a copy of the value? 
Thanks in advance for any insight!
			
Richard Jones suggested:
The CAMLparam/CAMLlocal/CAMLreturn/CAMLxparam macros are pretty simple 
to understand.  I suggest you take some simple code using these 
macros, run it through cpp, and have a look at what these macros 
actually generate.  You will be able to make your own (possibly 
non-portable) variations which update caml_local_roots etc. directly. 
If that is necessary. 
			
Bob Matcuk then asked and Chris King answered:
Short answer: your function is okay, but not for the reasons you 
state.  Long answer: 

> I guess the real question is: is it even necessary to worry about this? 
> The function, as I said, will only ever be called from other C 
> functions (who have already designated these values as being 
> params/local to themselves, assuming they are written correctly). 

This doesn't matter.  The purpose of CAMLparam/CAMLlocal is twofold: 
first, to declare that a particular value shouldn't be garbage 
collected (which, as you point out, the other C functions take care 
of).  Secondly (and more importantly in this case), they inform the 
garbage collector of the all roots which it may need to update in the 
event that it relocates a block.  If you don't use CAMLparam/CAMLlocal 
and a GC cycle occurs, there's a chance that a block which one of your 
values points to will be moved, and your value will become invalid. 

> I 
> seem to recall reading somewhere that if you write a function that 
> will only ever be called from other C functions that have already 
> registered the values (via CAMLparam/CAMLlocal), then it's 
> unnecessary to do it again. 

AFAIK this is incorrect (because blocks can be relocated).  The only 
time it is safe not to register values is if your code does not assume 
that the contents of a value are valid after a call which could trip 
the GC. 
> The function doesn't allocate any new 
> values either, so it shouldn't trip the GC anyway, right? 

True, false.  caml_callbackN executes arbitrary code, which may or may 
not trip the GC.  hash_variant and caml_get_public_method are 
questionable also (since they return values), but looking at the Caml 
source code, it seems that they are safe (but I don't think the docs 
guarantee this). 
(BTW you should use caml_hash_variant rather than hash_variant; the 
comment for caml_get_public_method in caml/mlvalues.h should probably 
be updated to this effect also.) 

> Should probably check calloc for success and maybe throw an exception 
> if it failed... 

You could do this with caml_stat_alloc and caml_stat_free (in 
caml/memory.h).  These are equivalent to malloc/free but throw Caml's 
out-of-memory exception if they fail.  However in this case, I would 
simply declare args as an array.  Otherwise, if the callback throws an 
exception, args will not be freed unless you explicitly catch 
exceptions via caml_callbackN_exn, free it, and then re-raise the 
exception. 
Note that if you have no control over the C functions higher up the 
call chain (say an external library which calls your function), they 
could exhibit similar problems if they are unaware of the possibility 
of your function raising an exception.  The best thing to do in such a 
case would be to return an error condition if possible, or at the very 
least, print a warning and return or exit gracefully (the functions in 
caml/printexc.h help here). 

> Which actually brings me to another quick question: if 
> I throw an exception, say caml_failwith("..."), is it necessary to 
> still call CAMLreturn after it? Or will the exception cause the 
> function to exit? 

The exception causes the function to exit.  You can see which 
functions act like this in the header files by looking for the 
"Noreturn" attribute at the end of their declaration. 
> Is it an invalid assumption that it is unnecessary to bother with the 
> CAMLparam/CAMLlocal stuff since there's nothing to trip the GC? If so, 
> what is the best way to handle all the CAMLparam/CAMLlocal stuff? 

Yes, it is an invalid assumption, because your code may in fact trip the GC. 
BUT 

Look over the function you wrote carefully.  Notice that values obj 
and *args are used only before the call to caml_callbackN, and that 
the value r is used only after that call.  Your function is indeed 
safe, only because you don't use after the "unsafe" call any value 
which was initialized before. 

Of course, for the sake of maintainability, I wouldn't in general 
endorse such eliding of CAMLparam/CAMLlocal.  I'd recommend putting a 
big ol' warning in there :) 

> For 
> example, CAMLlocalN(args, n + 1) is invalid because C does not allow 
> you to declare dynamic arrays. 

K&R C doesn't, but GCC does.  If you're using another compiler or some 
compatibility flag, then the alloca function (usually found in 
alloca.h) should do the trick.  It allocates space on the stack 
exactly like an array declaration does, so the guts of CAMLlocalN 
should apply to it. 
> I know values are just pointers so it is syntactically correct, but 
> what I'm asking is: is it safe to do? Should I be using some function 
> instead to create a copy of the value? 

Copying values with assignment is perfectly legal, provided the 
locations to which they are copied are registered with the GC first 
(just like any other value). 
Hope this was able to clear things up, I've hit many of these bumps 
myself while learning to write extensions.  The best thing to remember 
is that Caml's GC is not a reference counter but a generational 
collector and can move blocks from right under your nose.  Then the 
reasons to use CAMLlocal/CAMLparam become clear.
			

Using folding to read the cwn in vim 6+

Here is a quick trick to help you read this CWN if you are viewing it using vim (version 6 or greater).

:set foldmethod=expr
:set foldexpr=getline(v:lnum)=~'^=\\{78}$'?'&lt;1':1
zM

If you know of a better way, please let me know.


Old cwn

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.


Alan Schmitt