Sunday, January 17, 2010

Coding Styles and D

Well, I've been working on some D code of late...nothing too interesting except I'm working on a compression algorithm. I'll get to that when I figure it out, but one thing I did was I wrote some code to do RLE compression, which is very simple and very effective on the right types of files. However, I was noticing that the code wasn't running very fast and using an insane amount of memory. For a enwik8 file that was cut in half, it took 26 seconds to encode and that was my best effort. So, I asked for help as I thought there was something seriously wrong and I didn't know if it was my fault or a problem with D in either the compiler or the standard library.

Turns out, it's both.

What was happening is that I was creating a lot of small arrays and concatenating them, which isn't the most efficient way to do things. I was under the assumption that basic operations like that would be fairly efficient since they're part of the language. I was wrong. Apparently the garbage collector isn't quite up to snuff yet, so it didn't do it's job properly. So, I followed the advice that I got when I posted to the digitalmars.D.learn newsgroup (Posting here) and I got a tremendous speedup. After I was done with the changes, I managed to get the time down to less than 2 seconds for encoding and decoding!

Now, where do coding styles come into play? Easy: I was coding as if I was developing in Java or Perl vs. a systems programming language like C. In those languages, array/stream/whatever concatenation probably would have been the norm, at least that's the way it seems in Java when I work with strings, and you get acceptable performance. In C, you can't do that so you have to do things differently, specifically you have to handle all memory allocations manually. When I switched to something more similar, my code became much more efficient.

Boy, this makes me wish I did more C-level coding at work.

Thanks again to all those who helped me out with this little problem!

Labels: , , ,


Post a Comment

Links to this post:

Create a Link

<< Home