Saturday, July 11, 2009

XMPP gen_component update: Design and features

For the last few weeks I've been trying to put together a framework for creating XMPP components in Erlang. While the code I have now is not by any means covering all possible features the XMPP developer might need, I think it's a good starting point. I'm going to go over the interface and features of gen_component behavior that represents framework's API. Your input is very much appreciated.

gen_component implementation is backed by gen_fsm, so just like for gen_fsm, there is a certain set of callback functions that must be provided by a callback module.
XMPP (external) component could be seen as intermediary between XMPP server that routes messages, and some backend service that needs to be exposed to XMPP server's clients. The service could be anything: a database, a game, a stock ticker, Twitter feeds etc. Hence there are functions handling XMPP server conversations, such as handle_iq/4, handle_presence/4 and handle_message/4, and the functions handling interactions with backend service, such as feed_component/2 and handle_feed/2. XMPP service discovery support for the component is backed by functions disco_info/2 and disco_items/2. There are also functions that deal with life cycle of gen_component, such as start_component/6, stop_component/1 and their respective callbacks init/2 and terminate/1. Interface includs some helper functions such as send_packet/2 and print_packet/1.

As a proof of concept I built a test_component module that implements gen_component callbacks. test_component represents mocked weather alert service Jabber clients could subscribe to. The client browses available weather alerts and registers with ones he wishes to get updated with. That's how it looks like in action:

exmpp library has been used extensively in gen_component, and I found it pretty stable, although there were some gotchas that I attribute to my lack of experience. I'm planning to do a detailed code walkthrough with comments on exmpp usage.

Source code for gen_component is here.


Friday, July 3, 2009

gen_component project: service discovery with exmpp

The gen_component behavior is growing up and is making more sense to me as the XMPP component project is moving along. I can even see gen_component rivaling gen_server in the nearest future :-). Jokes aside, it will have handle_iq, handle_message, handle_presence and possibly more callbacks with helpers glueing it all together, in a fashion similar to OTP behaviors.
By the way, the component doesn't even have to be a process, depending on what exactly your component does. So far I managed to avoid maintaining conversational state, but will have to introduce it going forward. For instance, to tell the registered user from a new one, having state seems to be a must, doesn't it? Still, should I use Mnesia for it, rather than process state?

The project is now having service discovery written using exmpp. The imaginary weather alert service exposes itself through disco_info/1 and disco_items/1 (which are now part of gen_component).

That's how it looks like in Psi:


As you can see, there are two weather alerts running, each one having few nodes for specific weather alerts. Are you thinking of stock tickers, twits etc. ? Yes, everything is possible with gen_component and XMPP :-)

Check out the source code and instructions.
Next in line is in-band registration.
I'll be doing web interface with Strophe or jsJac as well once I have a chance.
I would be interested in your thoughts and ideas on gen_component.

Writing XMPP components in Erlang with exmpp

I'm going to show how to create an external XMPP component in Erlang. If you're in a hurry and familiar with ejabberd, you may want to go straight to the source code.
This mini-project started when I decided to build XMPP interface for the system that had already been completed. I chose external component over ejabberd module, because I wanted to avoid ejabberd dependencies and be able to use it with any Jabber server. As the system was written in Erlang, and I had already written gen_mods, Epeios seemed to be an obvious way to go, as it promotes turning standard ejabberd modules into XMPP components. Life turned out to be not so rosy though, after I had realized that module development in Epeios is deprived of features that makes writing gen_mods so convenient, namely hooks and handlers. Epeios site states that they are working on it, but for now you're totally on your own when it comes to handling all traffic routed to the component. Epeios still handles XEP-0114 connection protocol, but this alone wasn't enough to justify its usage.

So I decided to create a module with XEP-0114 implementation by borrowing some code from xmpp_component.erl in Epeios distribution.
The Epeios code (version 1.0.0) implies that:
  • the component has start/2 and stop/1, which is what gen_mod behavior requires;
  • start/2 creates a process and returns {ok, Pid}, which is a deviation from official ejabberd module development doc;
  • the process created by start/2 has to be able to handle messages of form {route, JID_From, JID_To, Packet};
  • to send a stanza on behalf of component, the module calls xmpp_component:send_to_server/1;
So it appears that in order to use standard ejabberd modules in Epeios, one should wrap them in a way that satisfies above requirements. As my goal was just ripping out some code pieces rather than reusing the whole thing, I felt that following gen_mod wouldn't bring much value, instead I wanted a behavior that would aid component writing in a more structured way.
Hence the gen_component behavior was created...
-module(gen_component).

-export([behaviour_info/1]).
-export([send_packet/1, component_name/0]).
% Return a list of required functions and their arity
behaviour_info(callbacks) ->
[{receive_packet, 3}, {send_packet, 1},
{component_name, 0}];
behaviour_info(_Other) ->
undefined.

% Helpers
send_packet(XmlString) ->
xep0114:send_to_server(XmlString).

component_name() ->
xep0114:component_name().

...and the code in xmpp_component.erl was altered accordingly in order to replace gen_mod by gen_component.

This left me with only one dependency - xmpp_component:start_xml_parser/1 directly loads platform-dependent expat_erl.so driver, location of which has to be defined in Epeios configuration file. I didn't want to tweak configuration and remember to choose suitable driver every time the component is being deployed.
Recently released exmpp library came to rescue. The exmpp build script automatically resolves the dependency on linked drivers that are used by XML parser. However, the interface of exmpp XML parser is significantly different from the ejabberd one that Epeios is using. While I was preparing myself for a next round of tweaking, I found that exmpp team had already taken care of this problem by providing adapters (in src/compat directory) that allow client code to use ejabberd XML parser interface with exmpp behind it. After some more coding I had a minimal component that was visible to external clients upon connection to ejabberd and could send and receive XMPP packets routed by ejabberd.

References:

Your comments and questions are welcome.


Update, July 2, 2009: Revamped the code to add more exmpp stuff; the framework now supports multiple components running on the same node (previously it was only possible to run a single component). Next will add service discovery support. exmpp appears to be quite handy, many helpers etc. Need to spend more time with it, though - lack of docs and comprehensive examples slows me down considerably.

Tuesday, June 23, 2009

Erlang for Java programmer - Design patterns, p.1

The Gamma et al. "Design patterns" book formulates common desing patterns that could be applied to any object-oriented language. When we move to functional language domain though, many OO design patterns simply don't apply. This is hardly surprising, because functional programmers by definition don't do OO design. Some of the problems that OO design patterns are supposed to solve still have their counterparts in functional programming.

We are going to walk through the book's pattern catalog and try to show a matching Erlang approach for the OO patterns, where possible.

Going forward, I will be referring to "Design patterns" book as DP. In (unlikely) case you'll want to follow, it'll be helpful to have this book handy.




Adapter (Wrapper)



Intention(DP): Convert the interface of a class into another interface clients expect. Adapter lets classes work together that couldn't otherwise because of incompatible interfaces.



Let's just substitute "class" to "function", and we'll have it for Erlang:



Convert the interface of a function into another interface clients expect. Adapter lets functions work together that couldn't otherwise because of incompatible interfaces.



Now, function's interface obviously being its arguments, in most cases we can easily create adapter for that function, which is another function having desired set of arguments.



Example:



Trying to stick closely to DP example, let's assume that we are working on drawing editor and want to implement text drawing capabilities in module textshape. The behavior that we have decided to use for all shape kinds contains bounding_box/1 function that returns bottom left and top right coordinates for any shape. So we are now concerned with task of writing bounding_box() flavour for text shapes. We have 3rd party module textview at our disposal, and we want to reuse it for our purpose. It happens that textview exports (in Java terms, has public) functions:



get_origin(TextView) that returns origin point of the text view in form of tuple {X, Y}, and

get_extent(TextView) that returns width and height of the text view in form of tuple {Width, Height}.



Our bounding_box/1 function then might look as follows:




# Calculates bounding box for text drawing;
# returns tuple {TopLeftX, TopLeftY, BottomRightX, BottomRightY}




bounding_box(TextShape) ->
TextView = convert_to_textview(TextShape),
{X, Y} = textview:get_origin(TextView),
{Width, Height} = textview:get_extent(TextView),
{X, Y, X + Width, Y + Height).









What about convert_to_textview/1 function? Not to worry, it should be easy. Remember, TextView and TextShape are not objects, but rather data structures, so conversion is normally just a matter of knowing what these structures are and matching parts of them. Below is a sample implementation, just to give an idea:




# In this case, we assume that TextShape is a list of properties, such as:
# [{text, "Desing patterns"}, {font, "Arial"}, {color, "red"}, .....];
# TextView is a dictionary with keys viewtext, viewfont, viewcolor, viewstyle etc.




convert_to_textview(TextShape) ->
lists:foldl(fun(Prop, Acc) ->
{Key, Value} = convert_prop(Prop),
case Key of
not_present ->
Acc;
_Other ->
dict:store(Key, Value, Acc)
end
end, dict:new(), TextShape).


convert_prop({"text", Value}) ->
{"viewtext", Value};
convert_prop({"font", Value}) ->
{"viewfont", Value};
%% Conversions for other properties
...................................
%% Properties that don't have counterparts in TextView structure
convert_prop({_, _}) -> not_present.










That's it, we have Adapter implementation blueprint for Erlang. Looks kinda trivial, don't you think? That's what you're gonna like once you start Erlang coding - many things are easier and definitely less verbose compared to Java.

Thursday, June 18, 2009

Erlang for Java programmer - power of functions

I have been using Java for more than 10 years now, and at the time I had come across Erlang, while I was excited about what Erlang has to offer, I was a bit worried on how to survive in absence of beloved OO features, such as abstract classes, polymorphism, inheritance etc. It turned out that these questions were somewhat irrelevant, instead I should have asked:



How the problems that OOP (Java, in particular) solves using feature "X" can be solved by functional language (Erlang)?




Polymorhysm vs. Higher-order functions



I'd like to show a practical example where Java programmer would use polymorphism, and Erlang programmer would use higher-order functions.



One of my Jabber/XMPP projects required message filtering based on some set of conditions. The customer wanted to implement some kind of censorship for their chat rooms. If incoming message fails any of those conditions, it has to be dropped. Let's assume that for this task we will maintain a list of pairs {condition, text}, where condition is a boolean string function that applies to the message text, for example the list could look like:

{"contains", "@$#*%!"}, {"equals", "This chatroom sucks"}, {"contains", "morons"}

etc.



So to pass the "appropriateness" test, the incoming message should fail each condition in a condition list.

Now, let's limit ourseleves to 2 functions (contains and equals) and see how we go about coding this in Java:




package com.teamhand.messaging.utils;



import java.util.ArrayList;

import java.util.List;



public class Censorship {

List <Condition> conditions = new ArrayList<Condition<();

public Censorship() {

super();

}



public void addCondition(Condition condition) {

conditions.add(condition);

}



public boolean checkMessage(String message) {

for (Condition condition : conditions) {

if (condition.check(message)) return false; // message has been censored

}

return true; // message has passed the censorship

}



public static void main(String[] args) {

ContainsCondition c1 = new ContainsCondition("@$#*%!");

EqualsCondition c2 = new EqualsCondition("This chatroom sucks");

ContainsCondition c3 = new ContainsCondition("morons");



Censorship c = new Censorship();

c.addCondition(c1);

c.addCondition(c2);

c.addCondition(c3);

String message1 = "I'm a good message";

String message2 = "Are you gonna listen, you morons?";

System.out.println(c.checkMessage(message1));

System.out.println(c.checkMessage(message2));

}

}



// Condition class

package com.teamhand.messaging.utils;



public abstract class Condition {

public abstract boolean check(String message);

}



//Implementation classes

//

//

package com.teamhand.messaging.utils;



// ContainsCondition class

public class ContainsCondition extends Condition {

private String template;

public ContainsCondition(String template) {

this.template = template;

}



@Override

public boolean check(String message) {

return message.contains(template);

}



}



// EqualsCondition class

package com.teamhand.messaging.utils;



public class EqualsCondition extends Condition{



private String template;



public EqualsCondition(String template) {

this.template = template;

}



public boolean check(String message) {

return message.equals(template);

}



}









The implementation is typical - we have a polymorphic list of Condition instances, whose classes implement condition-specific check() method.



Now, it's Erlang's turn:




-module(censorship).

-export([check_message/2, test/0]).



condition({"contains", Phrase}) ->

fun(MsgBody) ->

string:str(MsgBody, Phrase) > 0

end;



condition({"equals", Phrase}) ->

fun(MsgBody) ->

(MsgBody == Phrase)

end;



condition(Other) ->

throw({error, {not_supported, Other}}).



%% Returns false is message should be censored, true otherwise

check_message(Message, Conditions) ->

not lists:any(fun(Condition) ->

Condition(Message) end, Conditions).



test() ->

C1 = condition({"contains", "@$#*%!"}),

C2 = condition({"equals", "This chatroom sucks"}),

C3 = condition({"contains", "morons"}),

CondList = [C1, C2, C3],

{check_message("I'm a good message", CondList), check_message("Are you gonna listen, you morons?", CondList)}.





Note how condition/1 function returns another function, so in check_message/2 we have a list of functions, each one representing a condition. As you may have expected, the solutions are very similar, except the Erlang one bears significantly less code.



To conclude, this post is not to start another flame war on merits of languages, but rather for those OOP programmers who have decided to try Erlang and may need some help in finding common ground while exploring functional style.