mvz / happymapper Goto Github PK

Environment

JRuby: 9.4.3.0
nokogiri (1.15.3 java)
nokogiri-happymapper (0.9.0)

Description

My generated xml loooks like:

<?xml version="1.0" encoding="UTF-8"?>
<namespace:demo xmlns:namespace=URL>
  ......
<namespace:demo>

Ruby code:

require 'happymapper'

class Demo
  include HappyMapper

  register_namespace 'namespace', 'URL'

  tag 'namespace:demo'
end

puts Demo.new.to_xml

It works fine with MRI Ruby, but within JRuby, it reports following errors:

Unhandled Java exception: org.w3c.dom.DOMException: NAMESPACE_ERR: An attempt is made to create or change an object in a way which is incorrect with regard to namespaces.
org.w3c.dom.DOMException: NAMESPACE_ERR: An attempt is made to create or change an object in a way which is incorrect with regard to namespaces.
                 setName at org/apache/xerces/dom/ElementNSImpl:-1
                  <init> at org/apache/xerces/dom/ElementNSImpl:-1
         createElementNS at org/apache/xerces/dom/CoreDocumentImpl:-1
                    init at nokogiri/XmlNode.java:330
                   rbNew at nokogiri/XmlNode.java:277
                    call at nokogiri/XmlNode$INVOKER$s$0$0$rbNew.gen:-1
            cacheAndCall at org/jruby/runtime/callsite/CachingCallSite.java:446
                    call at org/jruby/runtime/callsite/CachingCallSite.java:92
               interpret at org/jruby/ir/instructions/CallBase.java:561
             processCall at org/jruby/ir/interpreter/InterpreterEngine.java:367
               interpret at org/jruby/ir/interpreter/StartupInterpreterEngine.java:66
        INTERPRET_METHOD at org/jruby/internal/runtime/methods/MixedModeIRMethod.java:128
                    call at org/jruby/internal/runtime/methods/MixedModeIRMethod.java:115
            cacheAndCall at org/jruby/runtime/callsite/CachingCallSite.java:446
                    call at org/jruby/runtime/callsite/CachingCallSite.java:92
                callIter at org/jruby/runtime/callsite/CachingCallSite.java:103
               interpret at org/jruby/ir/instructions/CallBase.java:558
             processCall at org/jruby/ir/interpreter/InterpreterEngine.java:367
               interpret at org/jruby/ir/interpreter/StartupInterpreterEngine.java:66
        INTERPRET_METHOD at org/jruby/internal/runtime/methods/MixedModeIRMethod.java:128
                    call at org/jruby/internal/runtime/methods/MixedModeIRMethod.java:115
                    call at org/jruby/runtime/Helpers.java:600
       callMethodMissing at org/jruby/runtime/Helpers.java:117
  finvokeWithRefinements at org/jruby/RubyClass.java:520
                    send at org/jruby/RubyBasicObject.java:1703
                    send at org/jruby/RubyKernel.java:2355
                    call at org/jruby/RubyKernel$INVOKER$s$send.gen:-1
            cacheAndCall at org/jruby/runtime/callsite/CachingCallSite.java:446
                    call at org/jruby/runtime/callsite/CachingCallSite.java:92
                callIter at org/jruby/runtime/callsite/CachingCallSite.java:103
               interpret at org/jruby/ir/instructions/CallBase.java:558
             processCall at org/jruby/ir/interpreter/InterpreterEngine.java:367
               interpret at org/jruby/ir/interpreter/StartupInterpreterEngine.java:66
               interpret at org/jruby/ir/interpreter/InterpreterEngine.java:76
        INTERPRET_METHOD at org/jruby/internal/runtime/methods/MixedModeIRMethod.java:164
                    call at org/jruby/internal/runtime/methods/MixedModeIRMethod.java:151
                    call at org/jruby/internal/runtime/methods/DynamicMethod.java:212
            cacheAndCall at org/jruby/runtime/callsite/CachingCallSite.java:456
                    call at org/jruby/runtime/callsite/CachingCallSite.java:195
     invokeOther5:to_xml at nokogiri.rb:11
             RUBY$script at nokogiri.rb:11
                     run at nokogiri.rb:-1
     invokeWithArguments at java/lang/invoke/MethodHandle.java:710
                    load at org/jruby/ir/Compiler.java:114
               runScript at org/jruby/Ruby.java:1276
             runNormally at org/jruby/Ruby.java:1193
             runNormally at org/jruby/Ruby.java:1175
             runNormally at org/jruby/Ruby.java:1211
             runFromMain at org/jruby/Ruby.java:989
           doRunFromMain at org/jruby/Main.java:398
             internalRun at org/jruby/Main.java:282
                     run at org/jruby/Main.java:227
                    main at org/jruby/Main.java:199

I guess that the user cannot send a node with a namespace before adding the definition of the namespace.
I applied a workaround in the to_xml method and it works:

def to_xml(builder = nil, default_namespace = nil, namespace_override = nil,
             tag_from_parent = nil)
    
    <code>

    root_tag = "#{tag_from_parent || self.class.tag_name}_"
    if root_tag.include?(':')
      namespace, tag = root_tag.split(':')
      namespace_name ||= namespace
      root_tag = tag
    end
    builder.send(root_tag, attributes) do |xml|

    <code>
end

Hi would it be possible to build a model that has a parent or surround_with notation?
Use case:

• I have class Foo
• It includes a list of Bars
• The XML representation should be surrounded by Root:

<Root>
  <Foo>
    ...
    <Bars>
      ...
    </Bars>
  </Foo>
</Root>

I wouldn't want to declare a class Root for this

It doesn't seem to be possible to write one HappyMapper class that can parse both this document, with a global namespace --

    <?xml version="1.0" encoding="UTF-8"?>
    <address xmlns="http://www.unicornland.com/prefix">
      <street>Milchstrasse</street>
      <street>Another Street</street>
      <housenumber>23</housenumber>
      <postcode>26131</postcode>
      <city>Oldenburg</city>
      <country code="de">Germany</country>
    </address>

-- and this document, with qualified names --

    <?xml version="1.0" encoding="UTF-8"?>
    <prefix:address xmlns:prefix="http://www.unicornland.com/prefix">
      <prefix:street>Milchstrasse</prefix:street>
      <prefix:street>Another Street</prefix:street>
      <prefix:housenumber>23</prefix:housenumber>
      <prefix:postcode>26131</prefix:postcode>
      <prefix:city>Oldenburg</prefix:city>
      <prefix:country code="de">Germany</prefix:country>
    </prefix:address>

-- even though the two documents are formally equivalent. This makes it hard to write mapping classes when you don't know whether the elements are going to be at the root or embedded in another XML document, or what namespace they'll use in that other XML document.

I managed to come up with the awful hack below, but

1. I'm pretty new to Ruby and even newer to Ruby metaprogramming, so it's probably even more awful than it needs to be,
2. It only supports a global namespace on the element and all its attributes, and
3. because the prefix-to-URI information is only available at parsing time, it resets the (class-level) @namespaces attribute with every call to parse(), which is less than ideal and could cause various kinds of weirdness.

    class NamespacedElement
      def self.inherited(base)
        base.include HappyMapper
        hm_parse = base.method(:parse)

        base.send(:define_singleton_method, :parse) do |xml, options = {}|

          doc = if xml.is_a?(Nokogiri::XML::Document)
                  xml
                elsif xml.is_a?(Nokogiri::XML::Node)
                  xml.document
                else
                  Nokogiri::XML.Document.parse(xml)
                end

          namespace_uri = base.namespace_uri
          if namespace_uri
            namespaces = doc.collect_namespaces
            prefix = namespaces.invert[namespace_uri.to_s]
            if prefix && prefix.start_with?('xmlns:')
              prefix = prefix['xmlns:'.length..-1]
              base.namespace prefix
            end
          end

          hm_parse.call(doc, options)
        end
      end
    end

Example

    class Address < NamespacedElement
      # note we no longer include HappyMapper, that's done by the superclass
      def self.namespace_uri
        URI('http://www.unicornland.com/prefix')
      end

      has_many :street, String, :tag => 'street'
      element :postcode, String, :tag => 'postcode'
      element :housenumber, String, :tag => 'housenumber'
      element :city, String, :tag => 'city'
      element :country, Country, :tag => 'country'

    end

It would be nice if there was a straightforward way to just set a namespace_uri and let the prefix be resolved at parse time.

As per ruby gems , the latest update to this gem is 0.5.8 pushed on Oct 29, 2012. Changelog mentions 0.5.8 on 12th Oct 2013 ?? I'm confused.

Is there a way to get the Nokogiri equivalent of #inner_html instead of #content from HappyMapper? Some of my XML tags contain embedded XHTML that I need to retain, but it is getting stripped out. I thought passing the raw: true option to elements might give the result I'm looking for, but it doesn't seem to be doing anything.

In Ruby on Rails 6, I have the following xml fragment:

<Tag>
    <ClassName>Codice Fiscale</ClassName>
    <xs:xsd xmlns:xs="http://www.w3.org/2001/XMLSchema">
        <xs:simpleType name="Codice Fiscale">
            <xs:restriction base="X4006E1450246">
                <xs:pattern value="\p{L}{6,6}[0-9LMNPQRSTUV][0-9LMNPQRSTUV]\p{L}[0-9LMNPQRSTUV][0-9LMNPQRSTUV]\p{L}[0-9LMNPQRSTUV][0-9LMNPQRSTUV][0-9LMNPQRSTUV]\p{L}"/>
                <xs:pattern value="\p{L}{6,6}\d\d\p{L}\d\d\p{L}\d\d\d\p{L}"/>
                <xs:pattern value="\d{11,11}"/>
            </xs:restriction>
        </xs:simpleType>
    </xs:xsd>
    <Multiplicity>
        <MinOccurrence>1</MinOccurrence>
        <MaxOccurrence>1</MaxOccurrence>
    </Multiplicity>
</Tag>

My Tag.rb model is the following:

class Tag

    include HappyMapper

    register_namespace 'xs', 'http://www.w3.org/2001/XMLSchema'

    tag "Tag"

    element   :class_name, String, tag: 'ClassName'

    has_one   :min_occurrence, Integer, tag: 'Multiplicity/MinOccurrence'

    has_many  :patterns, String, tag: 'xsd/xs:simpleType/xs:restriction/xs:pattern/@value', namespace: 'xs'

I can read and work with the XML, without problems, but when I go to write it back using to_xml, the resulting file has incorrect tags, as showed in the following fragment:

<Tag>
    <ClassName>Codice Fiscale</ClassName>
    <xs:xsd/xs:simpleType/xs:restriction/xs:pattern/@value>\p{L}{6,6}[0-9LMNPQRSTUV][0-9LMNPQRSTUV]\p{L}[0-9LMNPQRSTUV][0-9LMNPQRSTUV]\p{L}[0-9LMNPQRSTUV][0-9LMNPQRSTUV][0-9LMNPQRSTUV]\p{L}</xs:xsd/xs:simpleType/xs:restriction/xs:pattern/@value>
    <xs:xsd/xs:simpleType/xs:restriction/xs:pattern/@value>\p{L}{6,6}\d\d\p{L}\d\d\p{L}\d\d\d\p{L}</xs:xsd/xs:simpleType/xs:restriction/xs:pattern/@value>
    <xs:xsd/xs:simpleType/xs:restriction/xs:pattern/@value>\d{11,11}</xs:xsd/xs:simpleType/xs:restriction/xs:pattern/@value>
    <Multiplicity/MinOccurrence>1</Multiplicity/MinOccurrence>
    <Multiplicity/MaxOccurrence>1</Multiplicity/MaxOccurrence>
</Tag>

The tags appears as being flattened, and there are 2 error:

1. the correct structure is not replicated on output
2. invalid tags are generated, like <Multiplicity/MinOccurrence>

Matjis suggests the following:

You shouldn't use slashes in the tag option and Happymapper shouldn't allow slashes in the tag option.

But this is the only way I found to parse the XSD fragment, following an example in the https://github.com/instructure/happymapper/#custom-xpath reference manual, where slashes are used:

class Media
  include HappyMapper

  has_one :title, String, :xpath => 'gallery/title'
  has_one :link, String, :xpath => 'gallery/title/@href'
end

Any suggestion on how to get rid of the fragment is welcome!

I'm trying to parse the following xml:

<prefix:address location='home' xmlns:prefix="http://www.unicornland.com/prefix">
  <street>Milchstrasse</street>
  <street>Another Street</street>
  <housenumber>23</housenumber>
  <postcode>26131</postcode>
  <city>Oldenburg</city>
</prefix:address>

But I can't find a way to remove a namespace from the element. I though this:

element :street, String, :tag => 'street', :namespace => nil

would do it.

There are schemas (e.g. Sitemaps and ResourceSync) with elements that allow free mixing of elements or attributes from other namespaces using <xsd:any> or <xsd:anyAttribute>. (In fact the elements in the ResourceSync schema are only meant to be stuffed into a Sitemap document.)

Right now there doesn't seem to be a good way to model this in HappyMapper, though I might be missing something. (Possibly related to #49?)

Currently, you cannot do things like search this repo due to the fact that it is a fork. Since this is the maintained version of this project, please contact Github to make this its own project. https://help.github.com/articles/why-are-my-contributions-not-showing-up-on-my-profile/#commit-was-made-in-a-fork

If an element is CamelCased, its content always evaluates to nil:

pry(main)> foo = HappyMapper.parse("<foo><MessageId>f3a16e4c-b812-4934-a009-742ac51668b5</MessageId></foo>")
=> #<#<Class:0x00000002945e60>:0x000000029448d0 @message_id=nil>
pry(main)> foo.message_id
=> nil

Renaming the elements to lowercase yields correct content but since my input will always be camelcased, that's not an option.

class Address
include HappyMapper
tag 'address'
element :street, String
end
XML = "<address><street>Long St.</street><city>Chicago</city></address>"
a = Address.parse(XML)
a.to_xml
=> "<?xml version="1.0"?>\n<address>\n <street>Long St.</street>\n</address>\n"

It'd be nice if the elements that it doesn't understand could still be passed thru to the output. Otherwise, there's the danger of losing data.

I just cloned the repo and attempted to run bundle install. This failed with:

There was a LoadError while evaluating nokogiri-happymapper.gemspec:
  no such file to load -- nokogiri from
  /path/to/nokogiri-happymapper/nokogiri-happymapper.gemspec:1

This makes sense, because the first line of the gemspec loads lib/happymapper.rb, which first does a require 'nokogiri'. Which is not available, because the Bundle has not been required yet (in fact, it has not even been constructed).

A solution for this could be to put the version in a separate file and load only that file. Are you interested in a patch that changes that?

I would love to know the opinion of others that use this gem. Let's start a conversation about the validity of this feature and if it should exist, how it should behave.

I was thinking that we often times we likely don't need all the power of HappyMapper and should rely on some conventions to get most of the work done for us. So I started thinking about what that might look like and started to work on a branch for that feature:

HappyMapper.parse(XML)

With no classes or configuration you can parse the example XML with little effort:

address = HappyMapper.parse(ADDRESS_XML_DATA)
address.street # => Milchstrasse
address.housenumber # => 23
address.postcode # => 26131
address.city # => Oldenburg
address.country.code # => de
address.country.content # => Germany

It is important to be aware that this no configuration parsing is limited in capacity:

• All element names are converted to accessor methods with underscorized names
• All value fields are left as String types
• Determining if there is just one or multiple child elements is hard, so it assumes it is one until it finds another with the same name.

Currently on the anonymous-mapper branch there is working code that will parse through most of the content and do a decent job. It is not working well with regard to converting the content back to #to_xml.

The following happymapper model defines a default namespace for the top level element and removes the namespace in the child elements.

 class Address
    include HappyMapper

    namespace :prefix
    tag :address

    has_many :streets, String, tag: 'street', namespace: nil

    has_one :housenumber, String, namespace: nil
    has_one :postcode, String, namespace: nil
    has_one :city, String, namespace: nil
  end

This model will parse XML that looks like the following:

<prefix:address location='home' xmlns:prefix="http://www.unicornland.com/prefix">
  <street>Milchstrasse</street>
  <street>Another Street</street>
  <housenumber>23</housenumber>
  <postcode>26131</postcode>
  <city>Oldenburg</city>
</prefix:address>

However, this model will not persist the xml correctly when #to_xml is used. It instead generates xml of the following format:

<prefix:address location='home' xmlns:prefix="http://www.unicornland.com/prefix">
  <prefix:street>Milchstrasse</street>
  <prefix:street>Another Street</street>
  <prefix:housenumber>23</housenumber>
  <prefix:postcode>26131</postcode>
  <prefix:city>Oldenburg</city>
</prefix:address>

When persisting the default namespace is being applied.

I'm creating a XML serializer, that comes after a mapper and a validator, and they share the same methods, so a code method exists in both mapper and validator. To avoid manually creating every field, I'm using some metaprogramming to set the values on the XML serializer, when I use a wrap I cannot set the child elements.

I have the following the following class:

require 'happymapper'

class Country
  include HappyMapper

  element :code, String
  element :name, String
end

class Profile
  include HappyMapper

  element :country, Country
  element :name, String
end

class User
  include HappyMapper

  wrap 'PersonalInformation' do
    has_many :profiles, Profile
  end
  element :name, String
end

I need to access the elements of PersonalInformation so I've tried:

User.new.class.elements

=> [#<HappyMapper::Element:0x0000558f73aed438
  @method_name="PersonalInformation",
  @name="PersonalInformation",
  @options={:single=>true, :name=>"PersonalInformation"},
  @tag="PersonalInformation",
  @type=#<Class:0x0000558f73aee3d8>,
  @xml_type="element">,
 #<HappyMapper::Element:0x0000558f73aece20
  @method_name="name",
  @name="name",
  @options={:single=>true, :name=>"name"},
  @tag="name",
  @type=String,
  @xml_type="element">]

But since the PersonalInformation it's an element itself, I don't have a way to find it's child elements. Am I missing something?

When trying to parse an xml object that has an empty string"" and doesn't meet the nil? criteria for a DateTime class I get ArgumentError: invalid date. I noticed the original jnunemaker/happymapper had a rescue in the method that perform the parse. Is there any way we could possibly re-add that to the supported_types class? I can submit a PR to merge if you could review. Or if there is already a way this is dealt with please let me know

test_0001_returns the proper length(Summary.for_email::with cassette Summary.for_email):
ArgumentError: invalid date
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-happymapper-0.5.9/lib/happymapper/supported_types.rb:113:in `parse'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-happymapper-0.5.9/lib/happymapper/supported_types.rb:113:in `block in <module:SupportedTypes>'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-happymapper-0.5.9/lib/happymapper/supported_types.rb:74:in `apply'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-happymapper-0.5.9/lib/happymapper/item.rb:72:in `typecast'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-happymapper-0.5.9/lib/happymapper/item.rb:95:in `process_node_as_supported_type'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-happymapper-0.5.9/lib/happymapper/item.rb:38:in `block in from_xml_node'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-happymapper-0.5.9/lib/happymapper/element.rb:20:in `find'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-happymapper-0.5.9/lib/happymapper/item.rb:38:in `from_xml_node'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-happymapper-0.5.9/lib/happymapper.rb:410:in `block (3 levels) in parse'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-happymapper-0.5.9/lib/happymapper.rb:409:in `each'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-happymapper-0.5.9/lib/happymapper.rb:409:in `block (2 levels) in parse'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-happymapper-0.5.9/lib/happymapper.rb:396:in `map'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-happymapper-0.5.9/lib/happymapper.rb:396:in `block in parse'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-1.6.8.1/lib/nokogiri/xml/node_set.rb:187:in `block in each'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-1.6.8.1/lib/nokogiri/xml/node_set.rb:186:in `upto'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-1.6.8.1/lib/nokogiri/xml/node_set.rb:186:in `each'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-happymapper-0.5.9/lib/happymapper.rb:394:in `each_slice'
    /opt/rubies/2.3.1/lib/ruby/gems/2.3.0/gems/nokogiri-happymapper-0.5.9/lib/happymapper.rb:394:in `parse'
    /Users/sarsena/src/order_client/lib/order_client/summary.rb:127:in `for_email'
    /Users/sarsena/src/order_client/test/cases/integration_test.rb:235:in `block (3 levels) in <class:IntegrationTest>'

XML:
    <summary>
         <order-id type="integer">1726827</order-id>
           <placed-at type="datetime">2011-09-20T12:42:13-04:00</placed-at>
            <cancelled-at type="datetime">2011-09-20T15:19:23-04:00</cancelled-at>
            <cancellation-reason type="string">INTERNAL</cancellation-reason>
            <delivered-at nil="true"></delivered-at>
            <shipped-at nil="true"></shipped-at>

I am dealing with a very fickle XML parser that requires the XML namespaces to be registered in a certain order.

For example, this passes:

<Root xmlns:ns1="..." xmlns:ns2="...">
</Root>

While this fails:

<Root xmlns:ns2="..." xmlns:ns1="...">
</Root>

Would it be possible to output the namespaces in the order that they were registered?

When emitting xml, namespaces are inherited from parent elements. However, when parsing xml, this is not the case. I'm not sure whether this is a bug or a missing feature.

Consider the following example

Bundler.require :default, :development                                                                                                                                                         
require 'happymapper'                                                           

class Beverage                                                                  
  include HappyMapper                                                           

  attribute :name, String                                                       
end                                                                             

class CoffeeMachine                                                             
  include HappyMapper                                                           
  register_namespace 'prefix', 'http://www.unicornland.com/prefix'              
  namespace 'prefix'                                                            
  has_one :Beverage, Beverage, tag: 'Beverage'                                  
end                                                                             

machine = CoffeeMachine.new                                                     
machine.Beverage = Beverage.new                                                 
machine.Beverage.name = 'coffee'                                                

xml = machine.to_xml                                                            
puts xml # Visually inspect, see it is as expected                              

machine = CoffeeMachine.parse(xml)                                              
puts "machine.Beverage is now unexpectedly #{machine.Beverage.inspect}" 

which will print machine.Beverage is now unexpectedly nil.

The reason is that when it parses the Beverage element, it doesn't do a namespaced lookup, because it doesn't know the namespace it should use, as no namespace property has been set for the Beverage class.

A fix would be to change from_xml_node and process_node_with_default_parser, so that the else case in from_xml_node becomes

      else
        process_node_with_default_parser(node, namespace, :namespaces => xpath_options)
      end

and process_node_with_default_parser becomes

    def process_node_with_default_parser(node, namespace, parse_options)
      parse_options = options.merge(parse_options)
      parse_options[:namespace] ||= namespace
      constant.parse(node, parse_options)
    end

If this is a feature you want, say the word and I'll contribute a patch with some specs.

NoMethodError: undefined method `to_xml' for #Cleo::Result:0x007fb21389c110

module Cleo
  class Result
    include HappyMapper

    tag 'element'
    has_many :term, String, :tag => 'term'
    element :id, Integer
    element :name, String
    element :score, Float
    element :timestamp, Time
    element :title, String
    element :url, String

    alias :terms :term
  end
end

When methods is called on the object c = Cleo::Result:0x007fb21389c110

1.9.2p318 :011 > c.methods
=> [:!, :!=, :!, :<=>, :==, :===, :=, :JSON, :id, :send, :`, :acts_like?, :as_json, :binding_n, :blank?, :breakpoint, :capture, :class, :class_eval, :clone, :dclone, :debugger, :define_singleton_method, :display, :dup, :duplicable?, :enable_warnings, :enum_for, :eql?, :equal?, :extend, :freeze, :frozen?, :gem, :hash, :html_safe?, :id, :id=, :in?, :initialize_clone, :initialize_dup, :inspect, :instance_eval, :instance_exec, :instance_of?, :instance_values, :instance_variable_defined?, :instance_variable_get, :instance_variable_names, :instance_variable_set, :instance_variables, :is_a?, :is_haml?, :j, :jj, :kind_of?, :load, :load_dependency, :method, :methods, :name, :name=, :nil?, :object_id, :presence, :present?, :pretty_inspect, :pretty_print, :pretty_print_cycle, :pretty_print_inspect, :pretty_print_instance_variables, :private_methods, :protected_methods, :psych_to_yaml, :psych_y, :public_method, :public_methods, :public_send, :quietly, :require, :require_association, :require_dependency, :require_library_or_gem, :require_library_or_gem_with_deprecation, :require_library_or_gem_without_deprecation, :require_or_load, :respond_to?, :respond_to_missing?, :score, :score=, :send, :silence, :silence_stderr, :silence_stream, :silence_warnings, :singleton_class, :singleton_methods, :suppress, :suppress_warnings, :taint, :tainted?, :tap, :term, :term=, :terms, :timestamp, :timestamp=, :title, :title=, :to_enum, :to_json, :to_param, :to_query, :to_s, :to_yaml, :to_yaml_properties, :trust, :try, :unloadable, :untaint, :untrust, :untrusted?, :url, :url=, :with_options, :with_warnings, :y]

and sure enough there is no to_xml listed in the methods call. I was thinking that to_xml should be an instance method and not a class method.

https://github.com/dam5s/happymapper/blob/master/lib/happymapper.rb#L346

I could be wrong. I have been looking at xml files for 3 days.

--Cheers.

It would be great to be able to order the elements exactly like they are ordered in the class definition. It would help compare input to output if they were in the same order.

I have xml that looks like this (sometimes):

<event name="695 Mtrs (Or)" id="1744534" num="7" time="21:06:00" progressCode="X" pmsg="SIS: LIVE" placesExpected="2">
    < result id="3058272"></result>
    <nonrunner>
        <selection id="26408531" num="6" name="Vacant" status="V"></selection>
    </nonrunner>
    <selection id="26408530" num="1" name="Ted" status="V"></selection>
</event>

I have an event class:

class Event
  include HappyMapper
  namespace 'hrdg'

  has_many :non_runners, NonRunnersList
  has_many :selections, Selection
end

class NonRunnersList
  include HappyMapper
  namespace 'hrdg'

  tag 'nonrunner'

  has_many :selections, Selection
end

class Selection
  include HappyMapper
  namespace 'hrdg'
  tag 'selection'
end

Problem is that my nonrunner/selections end up in the same list as my runners (selection).

Putting :xpath => '/event' on :selections on the event stops any selections in that element getting parsed. Is that broken?

How can I get the same elements that are in 2 places in the tree to stay in the right places?

(Sorry I cannot get the formatting right on this!)

I dislike having to type too much and transformation form snake_case to PascalCase just doesn't seem too sensible (element :mime_type, String, :tag => 'MimeType'). Would it be possible to just add an attribute that takes a set of params (ie. what case to transform into) or a block/proc to handle the transformation?

Examples:

• element :mime_type, String, :tag_format => :pascal
• element :mime_type, String, :tag => Proc.new { |name| return "%f %s" % [5, name] }

Given a nested structure with multiple namespaces (nsa and nsb) like this:

<nsa:Outer>
  <nsa:Nested>
    <nsa:Inner>
      <nsb:attribute>value</nsb:attribute>

Outer is parsed correctly and Nested is put in, too. But while the element for Inner belongs to Namespace A, it's class/type is defined in Namespace B. This yields nil when trying to parse.

I wonder how I should specify namespaces on classes and elements here?

BigDecimal#new has been marked for deprecation for awhile and it looks like it finally happened in Ruby 2.7? This broke our configuration that used #new as the parser:

element :payment, BigDecimal, :tag => 'payment', :parser => :new

I have been able to register a new SupportedType and get the new syntax working:

module HappyMapper
  module SupportedTypes
    register_type BigDecimal do |value|
      BigDecimal(value) if value && !value.empty?
    end
  end
end

Is the right approach or am I missing some other way to solve this issue?

For this example:

require 'happymapper'

class Root
  include HappyMapper

  tag 'Working'
  element :name, String
  wrap 'mywraptag' do
    element :description, String
    element :number, Integer
  end
  element :code, String

  def description
    'wooo'
  end

  def code
    'ABC123'
  end
end

root = Root.new
root.number = 12_345

puts root.to_xml

I have the following output:

<?xml version="1.0"?>
<Working>
  <mywraptag>
    <number>12345</number>
  </mywraptag>
  <code>ABC123</code>
</Working>

but if I comment the description method, and set the attribute like I do for the number it will generate the wrap tag as expected.

In my case I'm using those defined methods to get some fields from database objects, so I must admit that this could probably be not the most common scenario. Even though as far as I understand if should work.

Given XML like the following:

  <qux>
    <foo id="0"/>
    <foo id="1"/>
    <baz>
      <foo id="2"/>
      <foo id="3"/>
    </baz>
  </qux>

Mapping this with happymapper, Qux.foo should be an array of size 2 containing only the <foo>s with IDs 0 and 1, but in fact it's an array of size 4 containing both those and the deeply nested <foo>s with IDs 2 and 3, as demonstrated below:

#! /usr/bin/env ruby

require 'happymapper'

class Foo
  include HappyMapper

  tag 'foo'

  attribute :id, Integer
end

class Baz
  include HappyMapper

  tag 'baz'

  has_many :foo, Foo
end

class Qux
  include HappyMapper

  tag 'qux'

  has_many :foo, Foo
  element :baz, Baz
end

xml = '
  <qux>
    <foo id="0"/>
    <foo id="1"/>
    <baz>
      <foo id="2"/>
      <foo id="3"/>
    </baz>
  </qux>
'

qux = Qux.parse(xml)

puts 'qux/foo'
qux.foo.each { |f| puts f.id }

puts 'qux/baz/foo'
qux.baz.foo.each { |f| puts f.id }

This should print

qux/foo
0
1
qux/baz/foo
2
3

but instead it prints

qux/foo
0
1
2
3
qux/baz/foo
2
3

In most cases the namespace only applies to the root element but outputting this sort of XML does not seem possible. For example, given this schema one must output the following:

<ns1:Transmission xmlns:ns1="https://ldex.limra.com/xsd/1.0/LDExBEM">
  <TransmissionGUID>1234106c-d731-466a-8235-3377985285yy</TransmissionGUID>
  <!-- ... -->
</ns1:Transmission>

Now How to achieve this? The namespace method is used "if a node and all its children are all namespaced elements".

The tag method does not accept a namespace argument like element method does. Specifying:

element :transmission_guid, String, :tag => "TransmissionGUID", :namespace => nil

or :namespace => false does not prevent the namespace provided to namespace from being output.

Does not seem possible to output a namespace only on the root element.

This is odd...

Right now, inheriting models from a superclass is awkward, because you're supposed to include HapppyMapper in each subclass to copy and inherit the attribute lists defined on the class

So you end up with this:

class Super
  include HappyMapper
end

# And for each subclass:
class SubClass < Super
  include HappyMapper
end

This will work but is awkward:

class Super
  include HappyMapper

  def self.inherited(base)
    super
    base.include(HappyMapper)
  end
end

HappyMapper should really define the inherited hook itself.

I have an XML document that contains & which if I parse using MyObject.parse(xml) returns an error Nokogiri::XML::SyntaxError (92:15: FATAL: xmlParseEntityRef: no name)

/app/vendor/bundle/ruby/2.6.0/gems/nokogiri-1.10.4/lib/nokogiri/xml/document.rb:62:in `read_memory'
/app/vendor/bundle/ruby/2.6.0/gems/nokogiri-1.10.4/lib/nokogiri/xml/document.rb:62:in `parse'
/app/vendor/bundle/ruby/2.6.0/gems/nokogiri-1.10.4/lib/nokogiri/xml.rb:35:in `XML'
/app/vendor/bundle/ruby/2.6.0/gems/nokogiri-happymapper-0.8.1/lib/happymapper.rb:307:in `parse'

But when I use the following code (https://github.com/mvz/happymapper/blob/master/lib/happymapper.rb#L307) to parse the same document there's no error and a Nokogiri::XML::Document document is returned.

doc = Nokogiri::XML(xml, nil, nil, Nokogiri::XML::ParseOptions::STRICT, nil)

I have tried escaping & with &, but still see the same issue.

def content(name)
@content = name
attr_accessor name
end

• Date
• Authors
• Minimum Ruby version
• Move Rake into gemspec?

I have follow XML:

<?xml version="1.0"?>
<a:feed xmlns:a="http://www.w3.org/2005/Atom" xmlns:os="http://a9.com/-/spec/opensearch/1.1/" xmlns="http://schemas.zune.net/catalog/apps/2008/02">
  <a:link rel="self" type="application/atom+xml" href="/v9/catalog/apps/27f2e0ef-5a0c-4dca-a383-b89f9485b9d2?os=8.10.12393.0&amp;cc=US&amp;lang=en-US"/>
  <a:updated>2015-04-14T12:58:30.328606Z</a:updated>
  <a:title type="text">Beach Buggy Racing</a:title>
  <a:id>urn:uuid:27f2e0ef-5a0c-4dca-a383-b89f9485b9d2</a:id>
  <a:content type="html">Beach Buggy is back with an explosive sequel! Drive into an action-packed, surprise-filled world of off-road kart racing mayhem. Race against a field of rival drivers, each with unique personalities and special abilities. Build a collection of crazy powerups, like Dodgeball Frenzy, Fireball, and Oil Slick. Unlock and upgrade a variety of cars, from dune buggies to monster trucks. Test your skills in 6 different game modes on 15 imaginative 3D race tracks, against a pack of tropical-loving rivals with a serious case of road rage!

This is the official sequel Beach Buggy Blitz, the free driving game with over 30 Million players worldwide. Fast, furious, fun and FREE, Beach Buggy Racing is a kart-racing island adventure for all ages.

&#x2022; &#x2022; GAME FEATURES &#x2022; &#x2022;

EXCITING KART-RACING ACTION
Utilize your driving skills and a collection of creative powerups to fight your way to the finish line. It&#x2019;s not just a great looking 3D racing game, it&#x2019;s an epic battle with spectacular physics-based gameplay!

COOL CARS TO CUSTOMIZE
Use your winnings to collect and upgrade a garage full of unique cars, from monster trucks to muscle cars to lunar rovers!

TONS OF AMAZING POWERUPS
Beach Buggy Racing crushes other kart racers with over 25 totally unique Powerups ... and more Powerups are coming!

15 SPECTACULAR RACE TRACKS
Explore dinosaur-infested jungles, lava-spewing volcanoes, beautiful beaches, and mysterious swamps. Each unique race track is packed with hidden shortcuts and surprises.

COLLECT A TEAM OF RACERS
Recruit a team of drivers to play with, each with a unique special power like teleportation, flaming fire tracks, and confusion spells.

PLAY THE WAY YOU WANT
Chose between multiple control options, and customize the 3D graphics settings to optimize your play experience.


&#x2022; &#x2022; CUSTOMER SUPPORT &#x2022; &#x2022;

If you encounter a problem running the game, please email us at [email protected]. Be sure to include the device you're using, OS version, and a detailed description of your problem.

For fast support on most common issues please visit:
www.vectorunit.com/support


&#x2022; &#x2022; MORE INFORMATION &#x2022; &#x2022;

Be the first to hear about updates, download custom images, and interact with the developers!
Follow us on Google+ at www.vectorunit.com/+
Like us on Facebook at www.facebook.com/VectorUnit
Follow us on Twitter @vectorunit.
Visit our web page at www.vectorunit.com</a:content>
  <iapCount>13</iapCount>
  <userRatingCountDistributions>
    <oneStarRatings>11</oneStarRatings>
    <twoStarRatings>3</twoStarRatings>
    <threeStarRatings>5</threeStarRatings>
    <fourStarRatings>11</fourStarRatings>
    <fiveStarRatings>70</fiveStarRatings>
  </userRatingCountDistributions>
  <sortTitle>Beach Buggy Racing</sortTitle>
  <releaseDate>2014-11-03T17:30:21.417000Z</releaseDate>
  <whatsNew>v1.2.3
- Fixed crash when initiating IAPs

v1.2.2
- Fixed low resolution issue
- Fixed championship continuation issue
- Fixed touch steering responsiveness near screen edge

v1.2.1
- Fixed localization to German, Spanish, Russian, and Trad Chinese

v1.2
- 3 new tracks:  Fire &amp; Ice, Aquarius, Spooky Shores
- Championship Mode - Win gold-plated cars!
- Quick Race Mode - Simply pick car/driver/track and race!
- Win Power-Ups with new Coco Loco mini-game 
- Localized to German, Spanish, Russian, and Trad Chinese
- New Achievements
- Lots of Tweaks/Tuning</whatsNew>
  <visibilityStatus>Live</visibilityStatus>
  <publisherUrl>http://www.vectorunit.com/beach-buggy-racing</publisherUrl>
  <publisher>Vector Unit</publisher>
  <privacyPolicyUrl>http://www.vectorunit.com/privacy</privacyPolicyUrl>
  <averageUserRating>8.499456</averageUserRating>
  <userRatingCount>1838</userRatingCount>
  <image>
    <id>urn:uuid:f960a7db-890a-4157-a47d-261b3499b096</id>
  </image>
  <screenshots>
    <screenshot>
      <id>urn:uuid:1ddcf43c-1176-473b-bf0c-1f30c65a12a5</id>
      <orientation>90</orientation>
    </screenshot>
    <screenshot>
      <id>urn:uuid:2a6cff2f-0790-4005-abaf-0b8dbb79ce31</id>
      <orientation>90</orientation>
    </screenshot>
    <screenshot>
      <id>urn:uuid:9980a307-3793-4c60-b20f-8c3853ab9c47</id>
      <orientation>90</orientation>
    </screenshot>
    <screenshot>
      <id>urn:uuid:4ba0b8ae-dfd8-4e11-b9d1-18920dbf6e04</id>
      <orientation>90</orientation>
    </screenshot>
  </screenshots>
  <doubleWideImage>
    <id>urn:uuid:19e41422-cde5-41d3-b5e1-963a1abf02d5</id>
  </doubleWideImage>
  <squareImage>
    <id>urn:uuid:aca3e40a-3846-4f74-be87-fd1639282c34</id>
  </squareImage>
  <categories>
    <category>
      <id>windowsphone.Games</id>
      <title>games</title>
      <isRoot>True</isRoot>
    </category>
    <category>
      <id>windowsphone.RacingAndFlying</id>
      <title>racing + flying</title>
      <isRoot>False</isRoot>
      <parentId>windowsphone.Games</parentId>
    </category>
  </categories>
  <tags>
    <tag>Independent</tag>
  </tags>
  <offers>
    <offer>
      <offerId>urn:uuid:6eba4bfa-e9bf-47a2-869b-aa75e6317e19</offerId>
      <mediaInstanceId>urn:uuid:23deb049-4f8b-4041-8d52-b11d0ca2db21</mediaInstanceId>
      <clientTypes>
        <clientType>WindowsPhone81</clientType>
      </clientTypes>
      <paymentTypes>
        <paymentType>Credit Card</paymentType>
        <paymentType>Mobile Operator</paymentType>
      </paymentTypes>
      <store>ZEST</store>
      <price>0</price>
      <displayPrice>$0.00</displayPrice>
      <priceCurrencyCode>USD</priceCurrencyCode>
      <licenseRight>Purchase</licenseRight>
      <expiration>2100-01-01T00:00:00Z</expiration>
    </offer>
  </offers>
  <taxString>plus applicable taxes</taxString>
  <backgroundImage>
    <id>urn:uuid:93e13edb-c38f-49cb-aeb5-896302bbe1f7</id>
  </backgroundImage>
  <publisherId>Vector Unit</publisherId>
  <publisherGuid>urn:uuid:9841e024-61d1-422d-87cc-61e8012632d3</publisherGuid>
  <a:entry>
    <a:updated>2015-04-14T12:58:30.328606Z</a:updated>
    <a:title type="text">Beach Buggy Racing 2015.127.2334.5274</a:title>
    <a:id>urn:uuid:23deb049-4f8b-4041-8d52-b11d0ca2db21</a:id>
    <version>2015.127.2334.5274</version>
    <payloadId>urn:uuid:0a683af8-0d83-4171-b7e0-10bc986f7252</payloadId>
    <skuId>urn:uuid:23deb049-4f8b-4041-8d52-b11d0ca2db21</skuId>
    <skuLastUpdated>2015-02-06T22:58:16.790000Z</skuLastUpdated>
    <isAvailableInCountry>true</isAvailableInCountry>
    <isAvailableInStore>true</isAvailableInStore>
    <isClientTypeCompatible>true</isClientTypeCompatible>
    <isHardwareCompatible>true</isHardwareCompatible>
    <isBlacklisted>false</isBlacklisted>
    <url>http://cdn.marketplacecontent.windowsphone.com/public/9cfb7496-3060-47d2-955b-c23369d9f778</url>
    <packageSize>92904030</packageSize>
    <installSize>95898624</installSize>
    <estimatedDownloadSize>92841984</estimatedDownloadSize>
    <packageFormat>AppXBundle</packageFormat>
    <packageFullName>VectorUnit.BeachBuggyRacing_2015.127.2334.5274_neutral_~_hvbhrzr8672s2</packageFullName>
    <isFramework>false</isFramework>
    <dependencyPackages>
      <dependencyPackage>
        <id>urn:uuid:5dec6302-3af3-4a4b-bc6e-d800779376b9</id>
        <skuId>urn:uuid:1076f542-8a6f-4cda-86e2-012bb248c9ef</skuId>
        <url>http://cdn.marketplacecontent.windowsphone.com/public/bdbd4abf-a3ab-433c-be54-0ec0278f8f49</url>
        <packageSize>645282</packageSize>
        <installSize>1544192</installSize>
        <version>12.0.30113.0</version>
        <packageFullName>Microsoft.VCLibs.120.00.Phone_12.0.30113.0_arm__8wekyb3d8bbwe</packageFullName>
      </dependencyPackage>
    </dependencyPackages>
    <isUniversal>true</isUniversal>
    <supportedPlatforms>Get once, download on compatible Windows devices too</supportedPlatforms>
    <clientTypes>
      <clientType>WindowsPhone81</clientType>
    </clientTypes>
    <supportedLanguages>
      <supportedLanguage>Deutsch</supportedLanguage>
      <supportedLanguage>English</supportedLanguage>
      <supportedLanguage>espa&#xF1;ol</supportedLanguage>
      <supportedLanguage>&#x440;&#x443;&#x441;&#x441;&#x43A;&#x438;&#x439;</supportedLanguage>
      <supportedLanguage>&#x4E2D;&#x6587;(&#x7E41;&#x9AD4;)</supportedLanguage>
    </supportedLanguages>
    <deviceCapabilities>&lt;hwCapability&gt;&lt;requirementType&gt;Resolution&lt;/requirementType&gt;&lt;id&gt;ID_RESOLUTION_HD720P&lt;/id&gt;&lt;string&gt;HD720P (720x1280)&lt;/string&gt;&lt;required&gt;true&lt;/required&gt;&lt;/hwCapability&gt;&lt;hwCapability&gt;&lt;requirementType&gt;Resolution&lt;/requirementType&gt;&lt;id&gt;ID_RESOLUTION_WVGA&lt;/id&gt;&lt;string&gt;WVGA (480x800)&lt;/string&gt;&lt;required&gt;true&lt;/required&gt;&lt;/hwCapability&gt;&lt;hwCapability&gt;&lt;requirementType&gt;Resolution&lt;/requirementType&gt;&lt;id&gt;ID_RESOLUTION_WXGA&lt;/id&gt;&lt;string&gt;WXGA (768x1280)&lt;/string&gt;&lt;required&gt;true&lt;/required&gt;&lt;/hwCapability&gt;&lt;capability&gt;&lt;id&gt;internetClientServer&lt;/id&gt;&lt;string&gt;internet connection&lt;/string&gt;&lt;disclosure&gt;Disclose&lt;/disclosure&gt;&lt;/capability&gt;</deviceCapabilities>
  </a:entry>
  <a:author>
    <a:name>Microsoft Corporation</a:name>
  </a:author>
  <isUniversal>true</isUniversal>
  <supportedPlatforms>Get once, download on compatible Windows devices too</supportedPlatforms>
</a:feed>

All elements with namespace a parsed as expected with follow definition:

require "happymapper"


class AppMapper
  include ::HappyMapper

  namespace "a"
  tag       "feed"

  element :updated,             DateTime
  element :title,               String
  element :id,                  String
  element :description,         String,   tag: "content"
  element :publisher_url,       String,   tag: "publisherUrl", namespace: nil
end

But publisher_url field wasn't parsed.

I tried different combinations of xpath, tag, namespace options, but neither is working.
When I load this document to Nokogiri. I was able to access element with follow xpath expression: /a:feed/xmlns:publisherUrl

xml = File.read("/tmp/27f2e0ef-5a0c-4dca-a383-b89f9485b9d2.xml")
doc = ::Nokogiri::XML(xml)
doc.xpath("/a:feed/xmlns:publisherUrl").text # => "http://www.vectorunit.com/beach-buggy-racing"

Versions:
ruby: 2.2.1
nokogiri: 1.6.6.2
nokogiri-happymapper: 0.5.9

Does anyone know whether it possible to map such xml using nokogiri-happymapper?

It would be great to add attributes to elements without having to generate a class for the element

This can best be shown in a spec:

require "spec_helper"

module Sheep
  class Item
    include HappyMapper
  end

  class Navigator
    include HappyMapper
    tag 'navigator'

    attribute :field, String
    has_many :items, Item, tag: 'item'
  end
end

describe "emptyness" do
  let(:xml) do
    <<-EOF
    <navigator>
      <items/>
    </navigator>
    EOF
  end

  it "returns an empty array" do
    navigator = Sheep::Navigator.parse(xml)
    navigator.items.should be_empty
  end
end

It has something to do with the has_many :items, Item, tag: 'item' definition. If you use has_many :unicorns, Item, tag: 'item' the result is, as expected, an empty array.

HappyMapper model generator

Currently the most tedious part of HappyMapper is attempting to model the XML data as a HappyMapper class. After that it becomes a fairly enjoyable experience. What if we had a method that could generate for you the source code from a specified XML input.

Within code you would write:

model_text = HappyMapper.model(ADDRESS_XML_DATA)
puts model_text

This would output:

class Address
  include HappyMapper
  has_one :street, String
  has_one :postcode, String
  has_one :housenumber, String
  has_one :city, String
  has_one :country, 'Country'
end

class Country
  include HappyMapper

  content :content, String
  attribute :code, String
end

Perhaps this would be most useful as a command-line application:

$ happymapper address.xml > address.rb

I get messed up unicode characters when parsing, ie. Liiketoimintayksikkö => Liiketoimintayksikk\303\266

Items can take several options. These are sparsely documented. Some only work together, but this is unclear from the documentation.

The spec file is incredibly useful documentation for new individuals using this gem. It would be great to break this file into multiple separate files describing each of the features.

Realized it this morning while working with this gem, that there is no reason that the name of the gem could also work with the require.

Was wonder that attributes with default value are not included in the list of attributes for xml building with #to_xml. Is there any ways to set default value and build xml from object?

attribute "sHouseValPe", Integer, default: 500000

The #to_xml method supports optional arguments for recursion. Using positional arguments for this makes their use unclear so they should be replaced with optional keyword arguments.

Ideally, the recursive version should not be exposed to end users.

I'm running into an issue where some UTF control characters are failing to parse and raising errors from Nokogiri. The reason is that happymapper uses STRICT parsing by default. If i feed the problem XML directly to Nokogiri using the DEFAULT_XML then it happily parses away and I get no issues. The offending UTF control characters are stripped out.

Here's where happymapper sets the parse options to strict:
Line 261

xml = Nokogiri::XML(xml, nil, nil, Nokogiri::XML::ParseOptions::STRICT)

I'm thinking it would be nice to provide a way to pass in nokogiri parse options when making a parse call on a HappyMapper object.

Maybe through the options hash or by registering a callback. Thoughts?