Rants about Lua
Par jd le mardi, décembre 30 2008, 11:23 - Free Software - Lien permanent
I've started using Lua some months ago, while looking for a more powerful way to configure awesome. At this time, around March 2008, Lua seemed to be the best language to integrate inside the core system of awesome.
I still think that Lua was a good choice, but after 8 months, it shows some important drawbacks.
I'll try to keep my explanation simple and to make you understand everything, even if you do not know Lua.
I refer here to Lua 5.1
Design flaws
Length operator
Lua has a length operator on its objects, known as #. It can be used to get the size of various objects.
return #"lol" 3
This operator works for table, string, etc… It's possible to define this operator by setting a __len meta-method on a userdata value.
The problem is that you cannot redefine it on string or table objects, see:
> a = { "hello", "world" }
> return #a
2
> setmetatable(a, { __len = function () return 18 end })
> return #a
2
Indeed, looking at the Lua core code:
case OP_LEN: {
const TValue *rb = RB(i);
switch (ttype(rb)) {
case LUA_TTABLE: {
setnvalue(ra, cast_num(luaH_getn(hvalue(rb))));
break;
}
case LUA_TSTRING: {
setnvalue(ra, cast_num(tsvalue(rb)->len));
break;
}
default: { /* try metamethod */
Protect(
if (!call_binTM(L, rb, luaO_nilobject, ra, TM_LEN))
luaG_typeerror(L, rb, "get length of");
)
}
}
continue;
}
You clearly see that tables and strings always use the internal length operator, never the __len meta-method.
That's for me a design problem, which will cause more trouble. We'll see later.
__index and __newindex metamethods
Lua defines two useful metamethods, which are __index and __newindex. Both can be set on a table or any other object. __index will be called upon each read access to an undefined key on an object, and __newindex upon each write access.
Example
> a = {}
> setmetatable(a, { __index = myindexfunction, __newindex = mynewindexfunction }) -- function are not defined, this is just an example
> a[1] = "hello" -- This will call __newindex metamethod
> return a[2] -- This will call __index metamethods
> return a[1] -- This will NOT call __index
The last line does not call __index meta-method because a[1] does exists. This is a problem when you want to use table as object, because sometimes you want to monitor access to the table elements.
This can be easily worked around using a proxy system: you don't store things in the table you manipulate, but in another table.
> a = {}
> realtable = {}
> setmetatable(a, { __index = myindexfunction, __newindex = mynewindexfunction }) -- function are not defined, this is just an example
Where meta-methods are something like:
function myindexfunction(table, key) return realtable[key] end
function mynewindexfunction(table, key, value) realtable[key] = value end
This way, our a table will always be empty, and realtable will have the data. At every read or write access to a, the meta-methods will be called. This is very convenient and widely used hack.
But this has serious drawbacks: as we saw before, the length operator (#) cannot be redefined on a table. That means #a will always be 0, and you cannot get the table length anymore, except by defining another method or a special attribute.
Also, Lua has several functions in the table library that are used to manipulate table in a easy way. The problem is that standard functions like table.insert or table.remove do raw accesses to the table. Meaning that if you do table.insert(a, 1, 1) it will insert the value 1 at the key 1 into a, without calling the __newindex meta-methods, breaking all your beautiful object-oriented model.
Another solution is to use a userdata object, like done in the Lua newproxy function (which is under-documented).
The problem is that it breaks all the other functions that are waiting for a table as argument, because they see a userdata, not a table. So this time table.insert is now more usable, which somehow fixes the problem, but not in the right way IMHO. However, this allows to use the __len meta-method.
Development model
The development model of Lua is, from my point of view, non-existent.
There is no public version control system repository available, so there's no chance to really contribute to Lua. It seems only a defined set of people work on it and therefore, the development system is very closed to my eyes, in comparison of usual projects.
Conclusion
I still think Lua is a good choice, because it is very easy to integrate into any C program, and to expand to fulfill your needs. However, some bad design choices were made, and the poor and closed development model chosen does not allow to have a good overview of the future of Lua.
This has been well stated by the authors themselves.
Commentaires
Why not fix the __len issue, then? It could be considered for inclusion in the next release, and would be a welcome patch until then. The Lua core is very good and easy to modify, so this sort of fix is simple.
You could just patch your Lua to use __len "properly." You could do the same for __index/__newindex, and you could redefine table.insert and friends to call metamethods.
As far as the development model goes, you're totally right; the Lua core team is very particular about what patches they accept. However, the community is very strong; I'm sure any patches you make would be welcome on the power patches page here (http://lua-users.org/wiki/LuaPowerP...) There's even one to fix your __newindex issue!
IMO, the development model situation that you describe is an absolute showstopper.
In addition, I read that Lua arrays are 1-based, which has been understood for decades to be the wrong way to go.
As far as I read on the list, some of the issues were discussed but the Lua core team decided that it was not worth changin this.
I remember at least Daniel Silverstone raising against one of this issue, I can't remember which one and I lost the link to the list archive.
Now I'm thinking about forking Lua to at least provide a alternative and open development model.
The __len metamethod issue is already fixed for 5.2, which you can find in various pre-releases (if you can't find one, just email the list as ask -- the developers usually will provide a snapshot on request).
Proxy tables will be far more useful in the future as well due to the __pairs and __ipairs metamethods being added.
So far the development model, the complaints are kind of moot. Just think of it as being like any other Open Source project where the developers do a lot of experimental work on local git/mercurial branches and in which only the core developers have write access to the FTP server. That's pretty much exactly what it is.
The developers frequently take patches from the community, as well as take suggestions for improvements.
Lua is small enough to just include in whole in a project's source tree, so if you feel the need to customize it... just do it.
It sounds like what you want is Ruby (well, actually, it sounds like you want a version of Ruby that's as fast as Lua, but the fact that you list shortcircuit optimizations as 'design flaws' leads me to think you're gonna keep chasing the wrong tree for a while).
Have you thought of any other languages? Python / Ruby / Lisp / Perl ?
Emacs has been embedding lisp for years
http://www.gnu.org/gnu/rms-lisp.htm...
Might I suggest ECL
http://ecls.sourceforge.net/
Depends on what your requirements are and how decoupled your implementation from the embedded language is.
People who spurn Lisp often live to regret it…
The point is not find another language, Lua is from far the easier to embed and to extend.
dogbot, seeing that some of the issues I raised are fixed in 5.2 version as said in another comment, I'm not sure that I'm wrong.
Mike said: "In addition, I read that Lua arrays are 1-based, which has been understood for decades to be the wrong way to go."
1-based indexing has never been considered "wrong." C uses 0-based indexing so that p[n] can be compiled as simply *(p + n). C was extremely influential in the design of the mainstream languages that followed it, namely C++, Java, and now C#.
There are some lousy languages that use 1-based indexing, like BASIC and Fortran. But, there are plenty of good ones too, like Lua, Smalltalk, and Pascal/Delphi. Although languages like Pascal and Lua make it your decision what to use.
The lack of a __len metamethod for tables is not a flaw. Actually metamethods are present to provide behaviour for undefined operations. This is true for all metamethods of Lua, with just two exceptions: __index and __newindex on tables (hence the second problem with these metamethods that you mention, directly coming from the fact that they are exceptions). If you want to improve consistency in Lua don't ask for __len to be called on tables, ask for __index and __newindex to be removed.
If you want custom data types, userdata have *all* metamethods.
Finally the development model is driven by some lead developpers, just like the Linux kernel is. If you want to submit a patch to Linux, it won't be accepted unless you gain confidence from some important developper. It's exactly the same with Lua. Except Lua codebase is so small it doesn't require a public version control system to manage patches.
"easy to integrate into any C program, and to expand to fulfill your needs"
ECL.
Embeddable Common Lisp.
But then again, there's already StumpWM.