I *have*, however, been meaning to get some more blog entries out which take a look under the hood (along the lines of “How long is a piece of string?”). It’s just a question of time and I’m already quite late on getting part 4 of my devzone.zend.com series out the door.

As far as where you should start with your JIT compiler, just take a page out of the opcode cache book and hook zend_compile_file to get the opcodes, do some voodoo to turn that into C code, then run that into libgcc to make happy little binaries. The principle is simple enough, it’s just that voodoo step in the middle that’ll drive you to seek truth at the bottom of a bottle some late frigid night…

In terms of understanding how the opcodes fit together, you ought to start by just pumping scripts into VLD and doing your best to make sense of the output. The opcodes are (by and large) named pretty intuitively, QM being the major exception: Hint, it’s to do with ternary expressions. Here’s a few files of interrest in the Zend/ directory:

zend_language_scanner.l – Turns source into tokens (e.g. “echo” => T_ECHO )

zend_language_parser.y – Turns tokens into expressions (e.g. T_ECHO T_STRING ‘;’ => zend_do_echo(&$1) )

zend_compile.c – Turns expressions into opcodes (e.g. zend_do_echo(znode *expr) => ZEND_ECHO )

zend_vm_def.h – Executes opcodes (e.g. ZEND_ECHO => zend_print_variable(op1) )

Note that all these files (with the exception of zend_compile.c) aren’t exactly C-source. zend_language_scanner.l, zend_language_parser.y, and zend_vm_def.h are pre-processed by flex, bison, and zend_vm_gen.php, respectively.

Your book on writing PHP Extensions is awesome! Any thoughts of extending it to include Zend Engine details?

Thanks.

Going-insane-wannabe-Zend-Engine-Hacker,

Jacob Santos

————-

There would be a greater need for writing extensions than for hacking the Zend Engine, but it might be good for a double length book. It would help make this long journey shorter. Although I do realized that asking is probably a slap in a face, since you didn’t have such material when you started hacking.

I’ll take this as a rite as passage then, until which time such material does become widely available in printed form. I did find a lot of functions that might help in implementing the JIT compiler, but I really do need to hack and see what each does. That is quite a long process and it usually leads to other areas of the Zend Engine. For that is another reason for the creation of this series. Once I learn one part, I can accept it in ease, or at least as much ease as one can get with a month or two of hacking.

To paraphrase someone I once knew(RIP), who was referring to the sendmail configuration file:

It’s like the Necronomicon… You are warned away, but you go regardless, hoping to learn unearthly secrets. If your
sanity survives, you spend the rest of your life conversing with znodes, running from refcounts, and striking fear into most heredocs. I don’t know whether it has driven me insane or revealed to me deep secrets about the universe.

Thanks! Yeah, I figured it would be easier and quicker to learn and work with the Zend Engine than it would be to build my own compiler. I still want to build my own compiler, but “getting my feet wet” is something I should do to better learn how compilers work, or more specifically how the Zend Engine works.

If I can’t adapt to the Zend Engine, then there is no way in hell I can call myself a professional or say that I’m ready for the professional world where I’ll be working with another person’s code all of the time. At the very least, I can say on my resume, that I can adapt well to alien code bases. Need to learn not to rewrite everything I don’t understand (my New Year’s resolution).

It also gets the little ideas that are floating around the inside of my head out in coherent form, so that I can follow and choose which idea is worth pursuing in the amount of time that I will have. Summer will be most interesting indeed, but I will also spend some time compiling theory and breaking the Zend Engine over the course of two months. Should be a fun side project.

My first step was going to be just jumping and using an library to allow JIT compilation for the Zend Engine. When I actually took a better look at the Zend Engine, the areas unrelated to PHP Extension writing, I got the idea to do this series. Comments are quite sparse and yeah, it might have went better for me if I had another two or three years of C/C++ experience.

Drowning is a better term I think to describe how over my head I am. I do have better respect for the people who understand and maintain the Zend Engine. Actually, some of the ideas I have are far too complicated for me to implement, even if I did understand the Zend Engine. Taking some posts that I had saved from last year and converting them to this format.

@Greg Beaver

Yeah, I’ve done that, except for IRC as I’m not fond of chatting. Wastes way to much time. I also would not like to waste their time either with pointless question after pointless question. The quest is to find the answers myself and if I stumbled, then ask. I may be a n00b, but I find it is better not to act like one.