Thursday, February 17, 2011

JSF2 - Benchmark datatable

In this article I will bench the datatables of 3 JSF2 components frameworks :
  • Icefaces 2
  • Primefaces
  • Richfaces

I will focus on efficiency : page size, ajax request/response size, server load, and not on features.



Test setup

The datatable will display a list of books with 3 columns : ISBN, author and title. The datable is binded to an ajax paginator which display books 15 by 15.

Dependencies version:
  • Icefaces 2.0
  • Primefaces 2.2.1
  • Richfaces 4.0.0M6
  • Mojarra 2.0.3
  • Jetty 7.2.2

No specific configuration is set.

JSF Fragments

Icefaces



Primefaces



Richfaces



Page size benchmark

I'm measuring the page size and the ajax response size with HtmlUnit. Creating an unit test with HtmlUnit is quite simple and I did not met issue requesting Icefaces, Primefaces and Richfaces for the purpose of this article.


Page size


Icefaces page size is huge, very huge. Icefaces and Richfaces don't used minified JS. Richfaces can gain 80KB by using JQuery min and be below 200KB.

All Primefaces JS are minified.

Ajax Response size



Richfaces Ajax Response



Icefaces Ajax Response


Primefaces Ajax Response



Icefaces and Richfaces send the paginator and the table. The table contains ids and classes on all cells. The Icefaces paginator block is quite big, more than the datatable block.

Primefaces sends only the table. The paginator is updated on the client side. The table contains ids and classes only on rows.
But what if the data changes on the server side (delete or add) ? An user action on the paginator doesn't update it. It seems to be bugged.

Knowing that I put an id only on the components i needed to update that means JSF autogenerate the other Ids by itself. So the question is, why does it do that when we don't need them?


Server load

I have created a simple unit test which executes 10 concurrent threads with a cookies manager.

Test setup : Core 2 Duo 3Ghz 64bits, 2GB dedicated to the JVM.

Average response time

  • Icefaces : 36.5ms
  • Primefaces : 13.8ms
  • Richfaces : 25.2ms

You can basically have three time more users with Primeface than Icefaces (considering it is growing lineary)

Because Icefaces OOM my JVM very quickly without a cookie manager I did a very simple calculation on session size. I took the JVM memory consumption and the number of requests done :
  • Icefaces : 7800 requests ~1.8GB : ~230KB
  • Primefaces : 9400 requests ~900MB : ~100KB
  • Richfaces : 9400 requests ~900MB : ~100KB


Conclusion

The page used for the test is quite simple and does not reflect a real page nor the global performance of these frameworks. To be complete the generic components (tab, tree...) have to be tested too in order to make a robust conclusion.

Primefaces has the best datatable implementation although buggy (I hope Primefaces 2.2 will correct all the issues). Icefaces has clearly the worst performance on datatable on all test. Richfaces is between Primefaces and Icefaces closest to Primefaces.

if a winner should be decided it would be Richfaces since I have been quite disappointed by Primefaces [1].

Dear Icefaces, Primefaces and Richfaces users, if you see some improvements please post a comment.

The project is available on github.


Thanks Mathieu for correcting the spelling of the article! ;)


[1] Post on Primefaces forum that is quite revelent :
I also hope that 2.2/2.3 focuses more on fixing up bugs and making the component library more useable in general. Once you start to use the components in PrimeFaces for things other than trivial showcase examples you quickly run into a lot of problems and limitations (I'm grateful for PrimeFaces, but thought I would provide a little constructive feedback).

28 comments:

  1. Hi Nicolas, I'm Cagatay Civici project lead of PrimeFaces.

    First great work on this benchmark test, it clearly shows how lightweight PrimeFaces is. Regarding datatable issues you've faced, please note that datatable is entirely rewritten from ground up between 2.1 and 2.2. You've used 2.2.RC2 not 2.2. final. There are currently around 10 issues regarding datatable 2.2.RC2 which will be fixed very soon. We're working extremely hard on maintenance of 2.2.RC2 to finalize 2.2.

    Thanks for your work on this benchmark test.

    ReplyDelete
  2. Ahoi Nicolas!

    Please note that Suns EL-2.2 impl performs a class.forName for _every_ EL invocation... So most of the performance is probably consumed up by your currently used el-impl.

    I've used a slightly patched juel version (http://github.com/struberg/juel) in the application I currently work on (https://tiss.tuwien.ac.at /curriculum/curriculumVersion.xhtml?studyCode=600&version=2001U.0 - please remove space, also using PrimeFaces btw ;) ) and I got down from 6 seconds to 400ms for 1300 rows!

    You might try it yourself. If the EL cost gets reduced, then the datatable performance would have a higher impact on the whole picture.

    LieGrue,
    strub

    ReplyDelete
    Replies
    1. the latest update of the project was two years ago, is recomendable to use that jars? thanks in advance!

      Delete
  3. Interesting comparation. I have projects using Icefaces and RichFaces, now I'm testing Primefaces and waiting it to be more stable. The coolest thing in Primefaces is the rich component library and the good performance.

    Strub pointed an important information about the el-impl. It could be nice to redo the benchmark using another el-impl implementation.

    ReplyDelete
  4. Hi Nicolas,

    Thanks for posting your bechmark. It is a great work!

    I would suggest that you comment about how you measured ajax request/response size and wich tools you used. This way anyone could reproduce your tests or extend them.

    In addition, I think it would be a good idea to share your project bechmark on GitHub so we could implement more tests.

    Thanks once again

    ReplyDelete
  5. @struberg nice, TU-Wien is making progress, not just on Eclipse since we held a DemoCamp there late 2009. Does the patch you mentioned apply to JSF2 or another Java EE code?

    ReplyDelete
  6. @all : Thanks for your comments

    @cagataycivici : I will look forward 2.2

    @struberg : Thanks for the tip. With 3000 rows displayed I get an improvment of ~7%. I do not get an improvment like yours. I will update the bench latter when I will have time.

    @João : Thanks for your suggesion, I will add the tools. I just use Firebug and Google development tools ;). I will attach the project latter.

    ReplyDelete
  7. Is it possible to post the three .war files so that the test can be reproduced exactly?

    In the case of ICEfaces, IDs are applied to many elements to allow for the case that they are updated individually (such as server-side data change or user edits of the table). Pagination results in wholesale update of the entire table so is a special case.

    It would definitely be good to look at this more closely to see what optimizations are possible (for instance, the paginator itself should not require such a large update with ICEfaces, only the styles on the changing labels should be updated).

    ReplyDelete
  8. I will post tomorrow the project. I have automated with unit tests the results.

    ReplyDelete
  9. The project is now on github https://github.com/nithril/JSF2-Benchmark

    ReplyDelete
  10. Do the same test with pagination, filter and order on columns and a datatable with many row (a least a few milions).
    And surprise...

    ReplyDelete
  11. Most of PrimeFaces components does not work properly on GAE. GAE demo don’t change for a half an year. GAE has especial problems and just follow JSF 2.0 is not enough. I had to switch to RichFaces 4 because they test all components for GAE, despite the fact that PrimeFaces looks better for a first glance. They don’t have static resources, mojarra is practically not functional. Little success is possible only with MyFaces, because they support JSF more precisely. I spent a lot of time in forum trying to use PrimeFaces. PrimeFaces is great, but to tell the truth he doesn’t test it against GAE.

    ReplyDelete
  12. How to use patched EL with maven?

    ReplyDelete
  13. I have downloaded and installed the project on my own internal repository.

    ReplyDelete
  14. Thanks for posting the test application. For this particular test, the ICEfaces measurements can be improved by removing icefaces-ace.jar (these components are not used in the test so there is no need to load the associated JavaScript) and by setting org.icefaces.autoid to "false" in web.xml (the page contains explicitly assigned IDs and does not require fine-grained ajax updates).

    ReplyDelete
  15. btw PrimeFaces works on GAE smoothly, examples:

    http://primefaces-rocks.appspot.com/ui/home.jsf
    or
    http://primefaces-gae-kickstart.appspot.com/main.jsf

    fyi.

    ReplyDelete
  16. Anonymous, do you extracted PF resources and added as static?

    ReplyDelete
  17. Anonymous!
    It is to late for me now. My project if finished with richfaces. I am very happy obout this component library.

    ReplyDelete
  18. It would be interesting to see how those libraries perform on huge data sets. Has someone done such a test?

    ReplyDelete
  19. Hi, just for future reference, ICEfaces 2.0.1 and 2.0.2 include several important optimizations that greatly reduce the number and size of JavaScript files being loaded with the page.

    I'd highly recommend retesting with latest versions.

    Thx.
    Ken Fyten
    VP Product Development, ICEsoft Technologies

    ReplyDelete
  20. We use PrimeFaces because RichFaces was not JSF 2 ready in 2009. PrimeFaces datatable is not bad, but I would take RichFaces for new projects (or even GWT) - they are more stable and you will get less headache. RichFaces' table has an interesting feature missing in PrimeFaces - each table row and any table cell are updatable separately. Updatable rows / cells are quite important for real-time applications with server-push technologies.

    ReplyDelete
  21. Hello,
    you might be interested as well to take a look at this IceFaces vs Primefaces vs RichFaces tutorial: http://www.mastertheboss.com/web-interfaces/365-primefaces-vs-richfaces-vs-icefaces.html

    ReplyDelete
  22. Thanks,
    Very interesting.
    I use RichFaces for this site : Presta-Expert, cherche un expert.
    It perfect for this kind of functionnal web site.
    I like it because it seems that it is more stable and reliable than the others.
    Antoine

    ReplyDelete
  23. Thank you Nicolas. It is very Nice analysis.
    Did you get chance to update the analysis using one of the latest Primefaces like 3.4.2 or later version?
    To generate dynamic column headings and row headings, which is better : Primefaces 3.4.x, or Tomahawk?

    ReplyDelete
    Replies
    1. Hello SreeRama,

      Sorry, there is no chance that I update this analysis. Since 3 month, I do not do JSF anymore. Feel free to fork the github project.

      In my opinion PF is above all the other.



      Delete