Updates from February, 2012 Toggle Comment Threads | Keyboard Shortcuts

  • Tomás Augusto Müller 2:09 pm on February 28, 2012 Permalink | Reply
    Tags: benchmark, , , , JIT, , ,   

    Benchmarking NeoQuotes (Jetty + JRubyRack) 

    To perform basic HTTP benchmarks against a web application, you can start with ab – Apache HTTP server benchmarking tool. speed

    If you use Ubuntu, the installation is very simple:

    sudo apt-get install apache2-utils

    I used this tool to benchmark NeoQuotes application, testing how Jetty with JRuby-Rack are performing against multiple concurrent requests.

    First, start up Jetty with mvn jetty:run or with the command defined in project Procfile (sh target/bin/jruby -S config/jetty.rb for NeoQuotes).

    Next, execute multiple times the following command:

    ab -n 10000 -c 25 http://localhost:8080/

    where n = number of requests and c = concurrency level.

    Is worth to look to the number of requests per second, that are increasing every time the benchmark is executed. This happens thanks to JIT!

    Below some results after running ab -n 10000 -c 25 http://localhost:55012/ three times in a commodity hardware (all tests were performed on the application homepage where there is no interaction with Neo4j and no cache mechanism is configured):

    First execution:

    Time taken for tests:   73.162 seconds
    Requests per second:    136.68 [#/sec] (mean)

    Second execution:

    Time taken for tests:   67.006 seconds
    Requests per second:    149.24 [#/sec] (mean)

    Third execution:

    Time taken for tests:   65.725 seconds
    Requests per second:    152.15 [#/sec] (mean)

    Also, take a look at Min and Max threads of Jetty QueuedThreadPool, at config/jetty.rb configuration.

    Depending of your hardware and your VM configuration,the more threads you configure into min and max, the worse your throughput gets when your VM starts to reach its resource limits.

     
  • Tomás Augusto Müller 9:41 pm on February 13, 2012 Permalink | Reply
    Tags: , heroku, , , , , , , voice interfaces   

    Talking with Neo4j Graphs 

    Hello!

    In this post I will be covering all main details regarding the development of my entry for the Neo4j Challenge.

    The main objective of this challenge is to create a Heroku-ready template or demo application using Neo4j. So, I neo4j challenge websitethought to myself: – what kind of application would be nice to show up in this contest?

    After many ideas, here it is!  neoquotes logo

    In short, the application is a Stock Exchange symbol lookup using Neo4j and your voice.

    Ok! Let me explain what I did.

    My mainly objective was try to combine all the joy of Ruby programming, with the power and ease of use provided by the awesome Spring Data Neo4j, Spring Framework and JVM, to this amazing piece of software called Neo4j which let me curious since 2011 in the Jim Webber talk at QCONSP conference of that year.qconsp 2011

    So, to start things up, JRuby comes in as a perfect fit. It also allowed me to use Sinatra to easily develop the user interface (both, voice interface and GUI). I choose Sinatra in this case because of it’s simplicity and as long the application stay small, it beat Rails (as said by @DHH).

    Jetty is another very nice (and old) project, which provides a Web server and javax.servlet container, plus support for Web Sockets, OSGi, JMX, JNDI, JASPI, AJP and many other integrations, and can be used to develop self contained applications. In this article from Heroku blog we have a nice introduction to create a Java Web Application using Jetty. Its integration with Maven was also another motivation point.

    Talking about Maven, possibly in POM.xml resides one of the trickiest parts of the application.

    After configuring all the dependencies, aspectj-maven-plugin, jetty-maven-plugin, let’s look to configuration of appassembler-maven-plugin and jruby-rake-plugin:

    jruby-rake-plugin:

    <executions>
        <execution>
            <id>install-bundler</id>
            <phase>process-resources</phase>
            <goals>
                <goal>jruby</goal>
            </goals>
            <configuration>
                <args>-S gem install bundler --no-ri --no-rdoc --install-dir vendor/bundle</args>
            </configuration>
        </execution>
        <execution>
            <id>bundle-install</id>
            <phase>process-resources</phase>
            <goals>
                <goal>jruby</goal>
            </goals>
            <configuration>
                <args>script/bundle install --without development:test</args>
            </configuration>
        </execution>
    </executions>

    As we can see, the plugin execution is responsible for download and install Bundler, then next execute the specified script witch lives in application script folder. This script basically starts Bundler client (pre-configuring ENV['GEM_HOME'], ENV['GEM_PATH'] and ENV['BUNDLE_GEMFILE']), and following the parameters, all gems except the ones for development and test, are installed.

    Last, but not least, the appassembler-maven-plugin:

    <configuration>
        <assembleDirectory>target</assembleDirectory>
        <extraJvmArguments>-Djruby.native.enabled=false -Djruby.cext.enabled=false -Djruby.compat.version=1.9 -Xmx256m -Xss2048k</extraJvmArguments>
        <programs>
            <program>
                <mainClass>org.jruby.Main</mainClass>
                <name>jruby</name>
            </program>
        </programs>
    </configuration>

    Above we have the configuration for the assemble goal execution. The trick here is to generate the executable with JRuby (org.jruby.Main) as main class. And you can also set extra JVM arguments, like the version of Ruby language to use.

    After this brief look into POM.xml, let’s see how Heroku starts Jetty. This is defined in the Procfile:

    web: sh target/bin/jruby -S config/jetty.rb

    So, that’s it! Now our app is running on Heroku Cloud! For more details take a look into /config/jetty.rb to see how Jetty is being started up and /src/main/webapp/WEB-INF/web.xml to see the JRuby-Rack configuration. Other technical details you can find at application readme file.

    Now, let’s talk about Neo4j.

    Your life is a huge graph.

    (…) and I think it can be handled by Neo4j. Back to 2011, Jim Webber let me very

    Ok... not quite a graph. But you get the idea!

    Ok... not quite a graph. But you get the idea!

    interested in Neo4j with his talk at QCONSP 2011. Maybe because in that year I was working on my final graduation project – a recommender system to support fundamental analysis of the Stock Market using natural language processing, and I never forgot of one thing that he said. More or less like this: “MapReduce processing between http requests can be very expensive! Some problems cannot be resolved with this approach.”. Well, I did this. And I must confess: a graph database would have changed a lot my project.

    So, when I discovered this challenge, I decided “- ok, now is the time to take a closer look on Neo4j.”

    First thing I did: accessed Neo4j download page and downloaded the 1.6 Community version.

    After that, the server installation. It is super easy. Just follow this instructions.

    So, what’s next step? Learn, learn and learn!

    I was able to make some initial experiments very fast after a dynamic reading of this *superb* online resource: Good Relationships: The Spring Data Neo4j Guide Book. Thank’s to Michael Hunger, David Montag, Andreas Kollegger by this material.

    Also, I learned a lot from the following links:

    And that’s it. This is the magic of open-source software. Have a doubt of how one given thing work? Go to source code and read it! All you need to get started with Neo4j is this. Trust me ; )

    Giving voice to a graph

    I used Arbor.js library to display the Neo4j data to the user. But how differently the information can be visualized? Through voice interfaces of course!

    As I already worked with Voice Interfaces some years ago (in 2008 actually, when I won the Sun Students Contest. More info here and here), this was not a mystery. Again I used VoiceXML markup language, and the free service provided by Voxeo. Refer to the links above for more information about VoiceXML.

    Who is talking with who? (a high level view of NeoQuotes architecture)

    Who is talking with who? (a high level view of NeoQuotes architecture)

    Sinatra eased this part even more, through it’s easiness of building API’s like applications. All vxml interfaces and grammars are bellow /app/views/voice folder, and there are only two voice routes in application.rb.

    To use the Voxeo service, is simple. You only need to create an account, and after that, set up a new application through Voxeo Application Manager, informing the voice entry point of your application (the full URL to your route. In my case to the get ‘/voice/index.vxml’ route).

    Note that your application need to be accessible by Voxeo server. So don’t forget to git push heroku master, before setting up your app on Voxeo.

    Finishing, you’ll get the numbers to your application. I always tested with Skype VoIP number. Works very well here in Brazil and is totally free.

    You can try calling to NeoQuotes! This is the Skype VoIP number: +990009369996189207 (to see all available numbers for NeoQuotes access the application and click on the respective top nav bar link).

    Important: When you spell the stock symbol, say one letter, and wait the voice repeat what you just said. And go on. When you finish, you can say “Okay!”, “Go!” or “Search!” for example. If your desired symbol have an ‘.‘ you have to say “dot”. The voice will not repeat this (will stay in silence). Just continue with the letters or numbers.

    Summing up

    This demo application contains an interesting stack of technologies, providing an environment to develop robust and testable Neo4j applications, with a lot of fun. Certainly much more can be done and improved. And you can help filling issues and submitting pull requests!

    Please, help voting in this application at Gensen repository or help with a Tweet!

    Thank you Neo4j for running this challenge! It was very fun to participate.

     
    • Jean 4:38 am on November 24, 2012 Permalink

      Thanks for sharing your demo, it is pretty exciting stuff.

  • Tomás Augusto Müller 9:44 pm on February 28, 2011 Permalink | Reply
    Tags: , ,   

    Encoding and JRuby on Rails 

    Studing JRuby and Rails I faced some encoding problems. The solution was very easy. First you must have to create your database using utf8 character set. In case you are using MySQL, just do the following:

    CREATE DATABASE `yourdatabasename` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
    

    or

    ALTER DATABASE yourdatabasename CHARACTER SET utf8 COLLATE utf8_general_ci;
    

    If you already have a database with records, read this post from @akitaonrails.

    In my case, I’m using the activerecord-jdbc-adapter gem, and connecting to the database through the MySQL JDBC Connector.

    At this time, I just have downloaded the JDBC driver (.jar file), and copied to $JRUBY_HOME/lib folder (if you are using RVM, find the JRuby installation folder using the rvm debug or rvm info command).

    Lastly, I just have set the following parameters into connection string (in database.yml file):

    jdbc:mysql://localhost:3306/yourdatabasename?characterSetResults=UTF-8&characterEncoding=UTF-8&useUnicode=yes

    Maybe this will be useful to someone.

     
c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel
Follow

Get every new post delivered to your Inbox.

%d bloggers like this: