Generating PDF with ODF templates

There are several ways of generating PDF output with Rails. Specifically with Ruby or better yet, on all frameworks and languages. Some techniques include templates written in LaTeX[1] or DocBook[2], while others require manual generation of the entire PDF. That, actually, is not a pragmatic solution for most real-world environments, nor does it follow the MVC separation principle. Background and Motivation At the beginning of 2007, I started working for FAPES (http://www.fapes.es.gov.br), a government foundation that incentives research, science and technology in Espírito Santo, the state where I live. I was given the job to create the site for our freshly created foundation. It was created altogether with SECT, the Secretary of Technology in our state, around 2005, without even a physical place available. Since it is a small foundation, I was responsible for everything related to IT, including network infrastructure, help-desk and programming. At the time, I was asked to provide maintenance to NOSSABOLSA, a web system implemented in ASP, which was (and still is) an important program of FAPES/SECT for financing studies in private colleges for students of low income families. I knew nothing about ASP or Rails at the time. My previous serious web programming experience (several years before this job) was with Perl. I didn’t work with web programming for lots of years after that. At the middle of 2007, I had to develop a new site for promoting an event we were organizing that needed a subscription system, along with lots of static information. Since I had a Linux server available, I didn’t have to implement it in ASP, which would take a lot of days and I only had a week for developing the new site. I started to then look at the web for available modern web frameworks, that allowed me to develop the website faster. I took a look at Perl frameworks, of course, J2EE, .Net, TurboGears, Django, Rails and a lot more. It became clear that Rails was the right choice. So I learned Rails, and developed the whole site in a week. Since it would be a temporary site, I didn’t mind trying a new framework at the time. I was so excited by Rails and specially Ruby, that, when we needed a new permanent site for FAPES, it was the logical choice. Shortly after the site creation, there was a need to generate some contracts based on input from web forms that could be printed and delivered to FAPES. There were lots of problems generating the contracts in HTML. CSS was not well thought for printing. It was difficult (if at all possible) to setup headers & footers and actually the print depends a lot on browser configurations and rendering engines. As a result, the printed versions would not follow a unique layout, which was a problem to us. So, I started thinking about PDF output. I read all the usual PDF generation techniques but none of them seemed to fit my needs. Proposed Solution Overview The problem was that there were a lot of types of contracts and they were a bit long, which would take a lot of time for preparing them without a good template system. I knew the juridic department was unable to give me LaTeX or DocBook-formatted contract models. They only knew MS Word, and I had to live with it. There was also not a lot of time for implementing contracts generation. It seems that when under pressure, we are extremely creative. Fortunately, I remembered that ODF was, actually, a XML file, along other files in a folder structure that were zipped in an ODF file. I extracted the file and took a look at a special file, called content.xml. Then I realized that it was possible (and pretty easy) to replace some special text templates with form fields submitted to the web server. It was also pretty easy to import MS Word documents in OpenOffice.org[3] and save them in ODT format. And I didn’t have to teach the juridic department any new document writing technique, such as Latex or DocBook. They could just use what they were used to. Additionally, there were some free tools that could convert odt to pdf, using OpenOffice.org in a “headless” environment, which meant that I could run it as a background daemon without even having a graphical environment installed. This daemon could be run in the same server as the application or in another dedicated server, if you prefer. Here is a possible usage for setting OpenOffice.org as a daemon, listening on port 3003: soffice -accept=”socket,host=localhost,port=3003;urp” -norestore -headless -invisible -nofirststartwizard& Then, any UNO[4] enabled software could convert any ODF file to any OpenOffice.org support output format, including PDF. For instance, one could use PyODConverter[5] for converting an odt document to pdf: /opt/openoffice.org3/program/python DocumentConverter.py document.odt output.pdf Just be sure to edit DocumentConverter.py, changing the port to reflect the one OpenOffice.org is listening at, since the port can’t be currently passed as a parameter. Following the above example, it means: DEFAULT_OPENOFFICE_PORT = 3003 The overall idea is illustrated on the following pictures: ODF template + HTML form Final generated output A Possible Implementation I was pretty happy for having found the solution to my problems and only needed some little time to implement the solution. This technique, actually, can be very easily implemented in any language. At least in Ruby, all seem easy to implement. Here is what I currently use for generating PDF contracts (save it to config/initializers/contract.rb): require ‘rexml/document’ module Contract   CONST_FIELDS = {‘DirectorName’ => ‘Name of Director’, ‘FAPES_Account’ => ‘Account Information’}   CONTRACTS_URL=’/contracts’; CONTRACTS_DIR = Rails.public_path+CONTRACTS_URL   ATTACHMENTS_DIR = “#{CONTRACTS_DIR}/attachments/”   OUTPUT_DIR = “#{CONTRACTS_DIR}/generated/”;  TEMPLATES_DIR = “#{CONTRACTS_DIR}/templates/”   class << self     # Save the template file at public/contracts/templates/scholarship.odt, then call:     # Contract::generate(‘scholarship’, {‘student_name’ => ‘Rodrigo Rosenfeld’}, ‘scholarships/1’)     # public/contracts/generated/scholarships/1.pdf will be created. Output is PDF link address.     def generate (template_file, fields, output_file, options={})       fields.merge!(CONST_FIELDS)       template_file = TEMPLATES_DIR+template_file+’.odt’       # replaces non-alphanumerics to underscore. Security is responsability from calling method.       output_file.gsub! /[^\\\/\w\.\-]/, ‘_’       pdf_output_filename = OUTPUT_DIR+output_file+’.pdf’       pdf_output_filename_temp = options[:attachment] ? OUTPUT_DIR+output_file+’_without_attachment.pdf’ :  pdf_output_filename       output_file = OUTPUT_DIR+output_file+’.odt’       output_dir = File.dirname(output_file); FileUtils.mkdir_p(output_dir)       Kernel.system “unzip -o #{template_file} content.xml -d #{output_dir}” or return nil       content_file = “#{File.readlines(“#{output_dir}/content.xml”)}”       # before replacing expressions, generate table templates       options[:tables].each {|t| content_file=generate_table(content_file,         t[:table_name], t[:line], t[:fields])} if options[:tables]       replace_expressions(content_file, fields)       File.open(“#{output_dir}/content.xml”, ‘w’) { |f| f.write content_file }       FileUtils.cp_r template_file, output_file       #pdf-converter is a script, that currently uses PyODConverter (DEFAULT_OPENOFFICE_PORT=3003):       #/opt/openoffice.org3/program/python /usr/local/bin/DocumentConverter.py $@       Kernel.system(“zip -j #{output_file} #{output_dir}/content.xml; pdf-converter #{output_file} #{pdf_output_filename_temp}”) or return nil       return merge_pdf(pdf_output_filename_temp, ATTACHMENTS_DIR + options[:attachment], pdf_output_filename) if options[:attachment]       pdf_output_filename.sub /\Apublic/, ‘’     end     def replace_expressions(str, fields)       # The pattern “#{student_name#U}” will be replaced by ‘RODRIGO ROSENFELD’       str.gsub!(/#\{(.*?)(#(.))?\}/) do         result = (fields[$1] or ‘’)         case $3           when ‘U’; result.mb_chars.upcase.to_s           when ‘d’; result.mb_chars.downcase.to_s           when ‘C’; result.mb_chars.capitalize.to_s           # lots of other formatters here for writing number at full length, as currency, etc.           else; result # doesn’t change         end       end     end     # this generates dynamic tables into ODF Templates. It is necessary to define a name for the table in OpenOffice.     # ‘line’ starts at 1.     # generate_table(content_string, ‘Items’, 2, [{‘n’ => 1, ‘item’ => ‘Desktop computer’}, {‘n’ => 2, ‘item’ => ‘Laser printer’}])     def generate_table(content_xml, table_name, line, fields)       document = REXML::Document.new(content_xml)       template_line = document.root.elements[“//table:table[@table:name=’#{table_name}’]/table:table-row[#{line}]”].to_s       document.to_s.sub(template_line, fields.collect {|f| replace_expressions(template_line.dup, f)}.join)     end     # returns url to merged file or nil if it couldn’t be generated     def merge_pdf(contract, attachment, output)       [contract, attachment].each {|f| return nil unless File.exist?(f)}       Kernel.system(“pdftk #{contract} #{attachment} cat output #{output}”) or return nil       output.sub(Rails.public_path, ‘’)     end   end end The implementation is not really important and I will not talk very much about it, since I am pretty sure a lot of plug-ins with better options and implementation will be developed using this technique. Most of the implementation is trivial to understand. Optimizations can be made, of course. For instance, it should not be necessary to unzip content.xml from the template before each conversion. I’ll just take some time to explain some parts that might not be obvious to all readers. template_line = document.root.elements[“//table:table[@table:name=’#{table_name}’]/table:table-row[#{line}]”].to_s document.to_s.sub(template_line, fields.collect {|f| replace_expressions(template_line.dup, f)}.join) It is intended to support simple table templates. Patterns should be written in one line, which will be replaced for several lines containing a collection of data taken from the web, such as Items to Purchase, or whatever. It is necessary to name the table (see table properties in OpenOffice) and tell the method which line should be used as template. First line is number one. There is also a helper method for merging PDFs, so that one special PDF can be attached at the end of the dynamic one if it is necessary. The implementation could be changed to accept other parameters for specifying a header and a footer PDF. The implementation uses pdftk[6] for merging them. As you probably noted, you would write ODT files with patterns such as “I, #{student_name#U}”, agree...”, that will be replaced by “I, RODRIGO ROSENFELD, agree...”, when “Rodrigo Rosenfeld” is submitted to the web server, in a form. Test Cases You should also write an unit test for assuring the output is correctly generated. Here is a possible unit test (test/unit/contract_test.rb): require ‘test_helper’ class ContractTest < ActiveSupport::TestCase   # This test is not definitive, since it doesn’t test the generated pdf content. But chances are good to be correct if content.xml is correct and   # the generated file is a PDF (starts with ‘%PDF’). However this doesn’t test if the merge is good, although it assures a lot of steps are working.   test ‘generate contract’ do     FileUtils::rm_rf Rails.public_path + ‘/contracts/generated/scholarships/test/’     assert_equal ‘/contracts/generated/scholarships/test/rodrigo.pdf’,       Contract::generate(‘scholarship’, {‘student_name’ => ‘Rodrigo Rosenfeld’, ‘address’ => ‘R. Nascimento Silva, 107’,     ‘total’ => 4090.49}, ‘scholarships/test/rodrigo’,     :tables => [{:table_name => ‘Items’, :line => 2, :fields => [{‘name’ => ‘Books’, ‘value’=> 90.49},       {‘name’ => ‘Airline Ticketings for International Congress’, ‘value’ => 4000.00}]}],     :attachment => ‘scholarships/contract_terms.pdf’)     assert FileUtils::identical?(Rails.root.to_s + ‘/test/fixtures/expected_content.xml’,       Rails.public_path + ‘/contracts/generated/scholarships/test/content.xml’)     f = File.new(Rails.public_path + ‘/contracts/generated/scholarships/test/rodrigo.pdf’)     assert_equal ‘%PDF’, f.read(4) # Is the output a pdf?   end end Conclusion Generating PDF from ODF templates proved to be pretty easy. Enjoy your free time, now that you can save a lot of it manually preparing PDF generation! Or use it for writing a good plug-in for PDF generation using this technique. :) Recently, a new plugin for ODF generation, written in Ruby, is available at http://github.com/sandrods/odf-report/tree/master. This plugin uses the idea presented in this article for ODF generation, using the rubyzip gem for zipping/unzipping, instead of launching an external program for this task. For readers interested in implementing a plugin for PDF generation in Ruby, I would recomend taking a look at this ODF generator and adapt it to integrate with OpenOffice.org and PyODConverter as demonstrated in this article.

2009-08-30T03:26:55Z

http://railsmagazine.com/forums/2/topics/68

3 Generating PDF with ODF templates

2009-12-24T20:50:45Z