[Python-ideas] Proposal: Moratorium on Python language changes
This proves my dictum “Every language evolves into Java or COBOL”.
For the sake of backward compatibility a hip language stop innovating. Then comes a new language and with feature this language has not implemented for a while for the sake of backward compatibility.
The new language claims that the hip language is dead and become the new hip language.
and the cycle goes on …
Python has various Functional Programming Tools including:
While working on my *super secret* application for Google App Engine. I realized i was using an operation over and over again which can not be efficiently performed by above tools(or at least i am not that expert in python yet!)
The operation is:
“select from a sequence, the first element that satisfy a certain condition”
Well i roll out my own function to perform such operation
def select(test, list):
"""
Select the first element from a sequence that
satisfy the given test function
- compare The test function should have following
signature def test(item): and must return a boolean
- list The List from which element need to be selected
"""
selected = None
for item in list:
if test(item) == True:
selected = item
break;
return selected
For example if we want to select the first number from a sequence that is divisible by 2 we have to write some code like this:
>>> list = [1,7,6,9,21,34] >>> select(lambda x: x % 2 == 0, list) 6
What are the other elegant ways of doing this in python or in your favorite language?
~TH
Recently I have to implement prefix searching for a project. The searching need to be scalable and the result are to be returned as quickly as possible. The front end for this search was a Ajax suggestion like search. The screen shot below shows the prefix searching in action.
Ok this pretty simple Ajax stuff, nothing fancy and nothing new. What I am going to discuss is the backend not the frontend for such a prefix searching component.
One of the most common implementation used for prefix searching is:
select * from some_table where some_field like ‘prefix%’
In other word put all the words to be searched in a table and use the like clause. This works and has reasonable performance too for small number of queries. But if you have continuous deletion and insertion of new search words then this solution does not scale well for large number of users.
So let put these words in memory for faster response time. How do we search prefixes if we have all the words in the memory?
Put them in a list and do a sequential search? Ok you guess that too slow….. Put them in a hash table? What key we are going to search in the hash table we have only part of the key (prefix)!!
You must agree prefix searching is a specialized problem which requires a special data structure that is geared toward prefix searching. You have read the title so you know the name of that data structure. Yes Radix tree is one of such data structure that is suitable for prefix searching. A radix tree, Patricia trie/tree, or crit bit tree is a specialized set data structure based on the trie that is used to store a set of strings.
Trie is a multi-way tree structure useful for storing strings over an alphabet. The idea is that all strings sharing a common stem or prefix hang off a common node. The elements in a string can be recovered in a scan from the root to the leaf that ends a string. All strings in the trie can be recovered by a depth-first scan of the tree.
Below is a trie for keys “to”, “tea”, “ten”, “i”, “in”, and “inn”.
You can consider a Radix tree a condense form of Trie, in which intermediate nodes are merged together to save space. Below is shown the insertion process in a Radix tree.
I wrote a simple Java implementation for radix tree for the prefix search application mentioned above. You can find the code at http://code.google.com/p/radixtree/
I like tree structures because of their recursive nature, looks really natural to me 
Most of the operation on Radix tree like insert, delete, find, etc can be easily implemented a recursive manner. Apart from the insert method another interesting method in implementation is visit() method it is there to help with other operation like delete, find and contains
/**
* recursively visit the tree based on the supplied "key". calls the Visitor
* for the node those key metches the given prefix
*
* @param prefix
* The key o prefix to search in the tree
* @param visitor
* The Visitor that will be called if a node with "key" as its
* key is found
* @param node
* The Node from where onward to search
*/
private void visit(String prefix, Visitor visitor,
RadixTreeNode parent, RadixTreeNode node) {
int i = 0;
int keylen = prefix.length();
int nodelen = node.getKey().length();
// match the prefix with node key
while (i < keylen && i < nodelen) {
if (prefix.charAt(i) != node.getKey().charAt(i)) {
break;
}
i++;
}
// if the node key and prefix match, we found a match!
if (i == keylen && i == nodelen) {
visitor.visit(prefix, parent, node);
} else if (node.getKey().equals("") == true // either we are at the
// root
|| (i = nodelen)) { // OR we need to
// traverse the childern
String newText = prefix.substring(i, keylen);
for (RadixTreeNode child : node.getChildern()) {
// recursively search the child nodes
if (child.getKey().startsWith(newText.charAt(0) + "")) {
visit(newText, visitor, node, child);
break;
}
}
}
}
Off course you can have better look at the implementation by visiting the online code repository at http://radixtree.googlecode.com/svn/trunk/RadixTree/
Update: I have remove the link to the old download distribution, Please download the latest code from http://code.google.com/p/radixtree/ As there are some bug fixes and improvements.
Just as i was coding on my “personal project”, i need a javascript customizable toggle button (replacement for traditional Checkbox). As being strongly in favor of code reuse i first tried the “Google exposed world”. But this time Google disappointed me, i could not find a decent toggle button script that i can simple drop in my project.
So i decided to roll up my own. As i was writing it i decided to make it a blog post too!
I decided to write a toggle button i such a way that i do not have to insert javascript code all over my pages. So i start with a ToggleButton class.
toggle.js
/**
* A Custom toggle/checkbox component
*
* div - The div that will be converted to toggle button
* check - the intial state of the toggle button (possible values ture/false)
* val - the value ssociated with this toggle button. This value is passed to the call back method.
* this way you can store any sort of infromation with the toggle button.
* callbk - the call back method that will be called when a toggle action is performed
* (method signature callback(val, check))
*/
function ToggleButton(div, check, val, callbk)
{
this.checked = check;
this.value = val;
this.callback = callbk;
// store the old inner text
var text = div.innerHTML;
div.innerHTML = "";
// create a element
var a = document.createElement("a");
a.href="";
// bind the current object to the mouse event
a.onmouseover = this.over.bind(this);
a.onmouseout = this.out.bind(this);
a.onclick = this.click.bind(this);
// create a element
var img = document.createElement("img");
if (check)
{
img.src = ToggleButton.IMAGE_NAMES[1];
}
else
{
img.src = ToggleButton.IMAGE_NAMES[0];
}
img.border = "0";
img.hspace = "0";
img.vspace = "0";
img.align = "absmiddle";
a.appendChild(img);
// create a Text node (for toggle button title)
var title = document.createTextNode(text);
a.appendChild(title);
div.appendChild(a);
// store the image withe toggle button
// because we need to later change(toggle) it on different events
this.image = img;
}
ToggleButton.prototype.over = function ()
{
this.image.src = ToggleButton.IMAGE_NAMES[this.checked + 2];
}
ToggleButton.prototype.out = function ()
{
this.image.src = ToggleButton.IMAGE_NAMES[this.checked * 1];
}
ToggleButton.prototype.click = function ()
{
this.checked = !this.checked;
this.image.src = ToggleButton.IMAGE_NAMES[this.checked * 1];
if(this.callback == Function)
{
this.callback(this.value, this.checked);
}
else
{
var code = this.callback + '(' + this.value + ',' + this.checked + ')';
eval(code);
}
return false;
}
// the default images for the toggle button
ToggleButton.IMAGE_NAMES = new Array();
ToggleButton.IMAGE_NAMES[0] = "images/togglebutton0.gif";
ToggleButton.IMAGE_NAMES[1] = "images/togglebutton1.gif";
ToggleButton.IMAGE_NAMES[2] = "images/togglebutton2.gif";
ToggleButton.IMAGE_NAMES[3] = "images/togglebutton3.gif";
And here is how we use it…
test.html
<!doctype html public "-//w3c//dtd html 3.2 final//en">
<html>
<head>
<title>Cutsom Toggle Button</title>
<script type="text/javascript" src="prototype.js"></script>
<script type="text/javascript" src="toggle.js"></script>
<script>
function initToggleButtons()
{
$$('div').each(function(item)
{
if(item.readAttribute('togglebutton') != null)
{
new ToggleButton(item, item.readAttribute('togglebutton') == 'true' ? true : false, item.readAttribute('togglevalue'), callme)
}
});
}
function callme(value, check)
{
$('value').innerHTML = "togglevalue=" + value + " :: checked=" + check;
}
window.onload = initToggleButtons;
</script>
</head>
<body>
<div togglebutton="true" togglevalue="11111">Hell of day</div>
<div togglebutton="false" togglevalue="22222">Bad bad!</div>
<div togglebutton="true" togglevalue="33333">not sure</div>
<div togglebutton="false" togglevalue="44444">what ever</div>
<div id="value"><div>
</body>
</html>
All the magic happens in the initToggleButtons. What it does is that it simply convert all div with togglebutton attribute into check boxes. I have used prototype.js to make this task a bit easy. There is also a method callme it is a callback and gets called when ever a toggle button is clicked. The value mention in the div and toggle status is provided to it. You can do a AJAX call here or what ever you want to do.
Oh and here is a screenshot
(Warning: I am not a big fan of Flex 2 or Flash in particular)
I was working on a small demo for a client. This demo was requested to be built with Flex 2. So i downloaded the SDK and starting hacking around.
When i end up with quit a few mxml file, compiling them start to become a headache. Because for the stupidest reason mxmlc does not compile multiple mxml files at once. Unfortunately i had to use a Windows based machine for creating particular demo.
As the application back end was in Java, so ANT was my automatic choice for build. But the problem was how to compile multiple files with mxmlc
It was simple task if i was on Linux box as i am quiet familiar with bash scripts. At this point i started to look at how to write advance script in windows bat files. More specifically i was looking for a some sort of loop structure that can iterate over all the files in a directory. In the end i end up with following bat file:
batch-mxlc.bat
for %%m in (%1\*.mxml) do call "%FLEX_HOME%bin\mxmlc.exe" -output "%2\%%~nm.swf" %%m
This bat file takes two arguments, first one is the input directory where your mxml files are present. The second parameter is the path to the output directory where you want your swf files to be generated.
The ANT script is simple and easy it just uses the exec task
<target name="help">
<exec executable="cmd">
<arg value="/c"/>
<arg value="batch-mxmlc.bat"/>
<arg value="${input.dir}"/>
<arg value="${output.dir}"/>
</exec>
</target>
Did you recognize what these icons are used for?
I bet majority of us must have recognize it. It is the so-called “Save Operation” icon. (always wonder what they save us from 
)
But what this image actually represents?
Yes the image represents a Floppy Disk. And the reason is that long a go floppy disk was the most commonly used visible storage media. That’s why it has become a standard image for save operation.
Wikipedia has a more detail entry.
For more than two decades, the floppy disk was the primary external writable storage device used. Also, in a non-network environment, floppies have been the primary means of transferring data between computers (sometimes jokingly referred to as Sneakernet or Frisbeenet). Floppy disks are also, unlike hard disks, handled and seen; even a novice user can identify a floppy disk (although this may change as they become less common). Because of all these factors, the image of the floppy disk has become a metaphor for saving data, and the floppy disk symbol is often seen in programs on buttons and other user interface elements related to saving files.
Although Dell still sell Floppy drives but Europe’s one of the largest computer store PC world store do not like floppies anymore.
But one thing is for sure Floppy drive does not represent the “primary external writable storage device” anymore.
So is not the time has come to change this Save icon image. Tango Desktop (help create a consistent graphical user interface experience for free and Open Source software) uses hard drive for the save action.
What you think should be the represented image for Save Action?
OR what other images you have seen for this action?
~TH
Today I am announcing the first public preview of Obneq. Obneq is a collaborative whiteboard. This is a preview release (early access) it does not represent the final shape of the application.
I have started working on Obneq couples of month back when I got bored working on back-end systems on my day job. So I decided to work on GUI intensive application in my free time. Yes this was a hobby project! Only very recently I got serious thinking to make it into a much useful product. So all sort of feedback is welcomed.
Obneq allow multiple people to work collaboratively on a whiteboard. And … and …
Well why not you just see the screen shot and decide yourselves!
Even better visit the web site www.obneq.com and discover yourself.
FAQ
Only a small group of people has access to this website for last couple of week and there are some frequently asked questions.
Why does Obneq mean?
There is a hint on the main page!
Why <put your favorite browser and version here> is not supported?
Huh? <put your favorite browser and version here> is a browser? Sorry I did not know 
OK now I know about if you are willing to help be out with cross browser issues and other logistic issues (like I do not have a Mac) I willing to support <put your favorite browser and version here>
Is this a Web 2.0 application? [The most funny question asked]
No! No! It is not a Web 2.0 application. It is a Web 3.0 application! Can not you see Obneq do not have any Beta/Alpha tag on its logo!
Why you have used ancient (dead) language like Java?!? [This question is the most burning question, I will write a complete blog entry about it]
Why I have not used the next cool languages Ruby, Python, Erlang, Lisp/Scheme, <put your favorite language here>, etc
Why I have touched this cursed/bloated/dead language Java?
First of all I am not a fan-boy of Java. I hate Java as much as a Ruby fan-boy does! But I also hate Ruby as much as Python fan-boy and I hate Python as much as …..
Truth to be told I have not found my love of life so far. I do have crush on Lisp a little bit. But I am still looking for love of life!
Will the service continue?
Definitely it will till I can afford the hosting.
Can I propose a new feature?
Off course you can. Flames, blames and suggestions are welcome.
In the last post I mention I mention about Garbage Collectors. This post talks about what can be done to make Garbage Collectors better for “serious programmers”.
The current garbage collector makes programmers hapless even when they know what sort of memory management will be good for them. Currently I have not seen a language that will provide garbage collection as well as control over the garbage collection. I could prefer the GC is a separated from the languages itself in such a way that it is pluggable. I can write a GC for my program if I want to otherwise it uses the default GC.
Every program is written to solve a problem. There can not be “generic program” that can solve all the problems. Similarly there can not a GC that can do memory management for all the programs. Memory management differs from application to application. So we sometime do need to take control of memory management. Sometime I just may want to take control of memory management for specific classes or data.
Are there any languages created are this line? Or Most of the people see Generic GC is enough for all problems?
When I used to program a “lot” in C/C++ one of the things that need to be take care of was memory management. I used to read article like the one on C++ User Journal about the “The Rule of Three” or rule of thumb like if you dynamically allocate memory you need the Rule of Three.
I was always told memory management is your responsibility.
The Pragmatic Philosophy
“Another key to their success is that they take responsibility for everything they do.”
The Pragmatic Programmer: From Journeyman to Master By Andrew Hunt, David Thomas
This approach of memory management is usually claimed as the root of all evil in these languages. Major errors are blame due to this and rightly so.
“This seemingly simple paradigm has been one of the major sources of programming errors. After all, how many times have you forgotten to free memory when it is no longer needed or attempted to use memory after you’ve already freed it?”
http://msdn.microsoft.com/msdnmag/issues/1100/gci/
Memory management STILL IS programmer’s responsibility
Then came Java/.NET with (false) claim that memory management is not programmers responsibility.
“In some programming languages, memory management is the programmer’s responsibility.
… …
An alternate approach to memory management that is now commonly utilized, especially by most modern object-oriented languages, is automatic management by a program called a garbage collector“
http:// java.sun.com/j2se/reference/whitepapers/memorymanagement_whitepaper.pdf
There are various Garbage Collection algorithms and each platform implements its own. Java for instance uses a Generational based Algorithm that uses different approach for each Generation of Objects. But the main method is to maintain a root set of statically allocated global objects. When times come for garbage collection each object in this root set is traverse to find all the objects that can be referenced from it. All the objects that can be references are added to an object graph. The objects that are not in the graph are one that can be garbage collected as there is no way to access them.
Off course this a simplified explanation as there are other problems like object references from one generation to another generation. Because each generation is garbage collected separately. So garbage collector has to do lot more than what I have said like maintaining references from one generation to another generation objects.
So I do not have to take care of memory management it is responsibility of Garbage Collector.
I do not trust Garbage Collector
I do not trust Garbage Collector and I have reasons for it. I have programmed in C/C++ and Java/.NET and most recently looking into Ruby and Lisp.
But I have seen
java.lang.OutOfMemoryError: Java heap space
and
System.OutOfMemoryException
errors more than memory corruptions and memory leaks using C/C++.
When you try to find out the reason for these exceptions. You are told that “you are not managing your memory properly”.
What!? All the developers/creator/fanboy of the languages like Java,.Net and Ruby all are lying to me! Memory management is STILL my responsibility.
Problem caused by this “lying”
Now due to lying about this important fact, everyone writing code in these languages never think about memory management. That’s the reason we see so many Out of memory problem in such languages.
Sometime back I was looking at the an application that was develop long time a go and now was having memory issues as the number of user has increased. The problem came out to be there were some thread specific object that deep down the object tree were still holding references to objects that were no longer used. So they could not be garbage collected. The problem was not visible when the number of users was small. But all hell broke as number of users increased.
When I asked programmer of that application why such things was ignored he said “does not this language take care of memory it self”?
What disappoints me is that even new languages that are built on top of experience from .NET and Java (for example most hyped language these days Ruby). There creator and supporter are still spreading the lies!
My request is to these people to stop spreading the lies and tell the programmers that:
“Memory management is still your responsibility”
Beyond Garbage Collectors … coming up in next post
Last night i was cleaning my 250GB Hard Drive and i found some old programs of mine These programs were written about some 5-6 years back when i was learning to program. These were my first “big” projects that i programmed (5 years back yes these were big project for me) They were not any real ideas rather they were just clone of some existing applications.
I ran the executable and wow i had some real flash back of those days when i was learning to program. One of the program was a clone of Microsoft Paint and the other one Breakout clone nothing original. But these are DOS programs! written in C++
These were the first project when i met the famous 640K barrier, when i try to allocate memory more than this limit.
Thanks to DOSBox i was able to got hold of some screen shots.
OK these screen shots does not look interesting but the point was not to show some ugly DOS screen shots. But rather i just wanted to mention how much fast technology has evolved. Right now even on my 2GB RAM box i feel i need more!
Is there going to be a limit to the want of memory? or are the Hard disk going to be fast enough in few decades that we will not be needing memory any more? or the other way around that memory become persistable and we do not need storage devices other than may be archiving.
At the moment having a headache and also have to take care of some deployments too i might talk more on this some time later.
Peace
TH
