appcues / exconstructor Goto Github PK
View Code? Open in Web Editor NEWAn Elixir library for generating struct constructors that handle external data with ease.
License: MIT License
An Elixir library for generating struct constructors that handle external data with ease.
License: MIT License
Hi,
what is the reason you decided not to do a check if the atom already exists?
https://github.com/appcues/exconstructor/blob/master/lib/exconstructor.ex#L155
I tried running this in my escript and go this error. Are there some dependencies I'm missing?
** (UndefinedFunctionError) undefined function Mix.Utils.underscore/1 (module Mix.Utils is not available)
Mix.Utils.underscore("field_four")
...
defmodule Bot.Mixfile do
use Mix.Project
def project do
[
app: :bot,
version: "0.0.1",
elixir: "~> 1.2",
escript: [main_module: Bot],
build_embedded: Mix.env == :prod,
start_permanent: Mix.env == :prod,
deps: deps,
default_task: "bot.run"
]
end
def application do
[
applications: [
:logger,
:nadia,
:httpoison, :poison,
:tzdata,
:exconstructor
]
]
end
defp deps do
[{:nadia, git: "https://github.com/zhyu/nadia"},
{:httpoison, "~> 0.8.0"},
{:poison, "~> 1.5.0"},
{:tzdata, "== 0.1.8", override: true},
{:timex, git: "https://github.com/bitwalker/timex"},
{:exconstructor, "~> 0.2.0"}
]
end
end
Problem parsing acronyms which are entirely uppercase within a string. Acronyms seem to be quite common is apis returning json.
What I mean by acronyms:
ImageID
--> image_url
, IPAddress
--> ip_address
Example
defmodule Identity.Endpoint do
defstruct [:admin_url, :region, :internal_url, :id, :public_url]
use ExConstructor
end
Identity.Endpoint.new(
%{
"adminURL" => "adminURL",
"id" => "id",
"internalURL" => "internalURL",
"publicURL" => "publicURL",
"region" => "region"
}
)
Returns
%Identity.Endpoint{
admin_url: nil,
id: "id",
internal_url: nil,
public_url: nil,
region: "region"
}
Screen for acronyms. Pull request on its way. ๐จ
It will be nice if you can pass a few structs and get something like:
from
%{"Command" => "string", "Created" => 1459328030, "HostConfig" => %{"NetworkMode" => "default"}}
to
%Container{command: "string", created: 1459328030, %HostConfig{network_mode: "default"}}
if it possible. Maybe from nested module:
defmodule Constructor do
defmodule Container do
defstruct command: 1, created: 2, host_config: 3
use ExConstructor
end
defmodule HostConfig do
defstruct network_mode: 1 #(or 4)
use ExConstructor
end
end
warning: negative steps are not supported in String.slice/2, pass 1..-1//1 instead
(elixir 1.16.0) lib/string.ex:2368: String.slice/2
The change is as I can understand and conclude from testing backwards compatible.
Last commit was quite one year ago ?
It would be a useful extension to be able to produce maps with keys cased in one specific flavour.
One example use would be to produce Javascript-style camelCase API responses while maintaining structs and maps internally in snake_case.
Tracked it down to the line underneath the unquote. Any ideas?
def unquote(constructor_name)(map_or_kwlist, opts \\ []) do
ExConstructor.populate_struct(
%__MODULE__{},
map_or_kwlist,
Keyword.merge(@exconstructor_default_options, opts)
)
Hi! Very nice library but i've got problem. How can i use exconstructor if data structure is map
, key type is string
and key format is like GlobalIPv6Address
?
For example, here what i get from docker remote api
%{"Command" => "string",
"Created" => 1459328030, "HostConfig" => %{"NetworkMode" => "default"},
"Id" => "string",
"Image" => "string",
"ImageID" => "sha256:string",
"Labels" => %{"build-date" => "2015-12-23", "license" => "GPLv2",
"name" => "CentOS Base Image", "vendor" => "CentOS", "string" => ""},
"Names" => ["/boring_varahamihira"],
"NetworkSettings" => %{"Networks" => %{"bridge" => %{"Aliases" => nil,
"EndpointID" => "string",
"Gateway" => "172.17.0.1", "GlobalIPv6Address" => "",
"GlobalIPv6PrefixLen" => 0, "IPAMConfig" => nil,
"IPAddress" => "172.17.0.2", "IPPrefixLen" => 16, "IPv6Gateway" => "",
"Links" => nil, "MacAddress" => "02:42:ac:11:00:02",
"NetworkID" => ""}}},
"Ports" => [%{"IP" => "0.0.0.0", "PrivatePort" => 80, "PublicPort" => 80,
"Type" => "tcp"}], "Status" => "Up 3 weeks"}
Wasn't trivial, at least to me, to set this working. Can I submit a PR to add to docs?
defmodule CreateShift, do: [defstruct([:shift_id]), use(ExConstructor)]
Would be nice to have something akin to the
%{"nestedField" => %{id: 1}}
|> Poison.decode! as: %Model{nested_field: [%Nested{}]}
@gamache hi! First of all, thanks for the library. Maybe I am mistaken, however since we are:
https://github.com/appcues/exconstructor/blob/master/lib/exconstructor.ex#L158-L159 from a newly generated string from Macro.underscore/1
and Macro.camelize/1
, wouldn't that be new strings associated with newly created atoms?
For example:
iex(141)> Regex.named_captures(~r/index_table:atom_tab\n*\n.*\n.*\nentries:(?<atoms>.*)/m, :erlang.system_info(:info))
%{"atoms" => " 20963"}
iex(142)> camelized = Macro.camelize("helloworld2")
"Helloworld2"
iex(143)> Regex.named_captures(~r/index_table:atom_tab\n*\n.*\n.*\nentries:(?<atoms>.*)/m, :erlang.system_info(:info))
%{"atoms" => " 20963"}
iex(144)> String.to_atom(camelized)
:Helloworld2
iex(145)> Regex.named_captures(~r/index_table:atom_tab\n*\n.*\n.*\nentries:(?<atoms>.*)/m, :erlang.system_info(:info))
%{"atoms" => " 20964"}
iex(146)>
I already checked issue #5 -- however I might have missed something?
Thank you!
I'm not sure if I'm missing anything but I can't see the point of this library.
Native: struct = struct(TestStruct, map)
Exconstructor: struct = TestStruct.new(map)
Problem parsing acronyms which are entirely uppercase within a string. Acronyms seem to be quite common is apis returning json.
What I mean by acronyms:
ImageID
--> image_id
, IPAddress
--> ip_address
Example
defmodule Identity.Endpoint do
defstruct [:admin_url, :region, :internal_url, :id, :public_url]
use ExConstructor
end
Identity.Endpoint.new(
%{
"adminURL" => "adminURL",
"id" => "id",
"internalURL" => "internalURL",
"publicURL" => "publicURL",
"region" => "region"
}
)
Returns
%Identity.Endpoint{
admin_url: nil,
id: "id",
internal_url: nil,
public_url: nil,
region: "region"
}
Screen for acronyms. Pull request on its way. ๐จ
defmodule Point do
@enforce_keys [:x, :y]
defstruct [:x, :y, :z]
use ExConstructor
end
Error when defining module:
** (ArgumentError) the following keys must also be given when building struct ExConstructorValidatorTest.Point: [:x, :y]
Could be caused by this:
%__MODULE__{},
Perhaps create struct with Kernel.struct/2
which does not check @enforce_keys
like %_{}
and Kernel.struct!/2
do:
Kernel.struct(__MODULE__, [])
iex(5)> TestStruct.new(%{"field_one" => "a", "fieldTwo" => "b", :field_Three => "c", :fieldFour => "d"})
%TestStruct{field_four: "d", field_one: "a", field_three: 3, field_two: "b"}
where does it get the 3 from?
Unless I'm missing something, it looks like this calls String.to_atom/1
on maps that could come from url params, which, given that atoms are never garbage collected exposes a denial of service vulnerability. Maybe I'm missing some validation somewhere, but it seems like this would need to verify that each key string is actually a key and never turn user input into atoms.
See: https://github.com/appcues/exconstructor/blob/master/lib/exconstructor.ex#L155-L156
This:
/app # iex -S mix
Erlang/OTP 21 [erts-10.3.5.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]
Interactive Elixir (1.7.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> defmodule TestStruct do
...(1)> defstruct field_one: nil,
...(1)> field_two: nil,
...(1)> field_three: nil,
...(1)> field_four: nil
...(1)> use ExConstructor
...(1)> end
{:module, TestStruct,
<<70, 79, 82, 49, 0, 0, 7, 224, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 249,
0, 0, 0, 23, 17, 69, 108, 105, 120, 105, 114, 46, 84, 101, 115, 116, 83, 116,
114, 117, 99, 116, 8, 95, 95, 105, 110, ...>>, :ok}
iex(2)>
nil
iex(3)> TestStruct.new(%{"field_one" => "a", "fieldTwo" => "b", :field_three => "c", :FieldFour => "d"})
%TestStruct{field_four: nil, field_one: "a", field_three: "c", field_two: "b"}
Expect: %TestStruct{field_one: "a", field_two: "b", field_three: "c", field_four: "d"}
Kernel.struct!/2 is stricter (see docs), which will prevent the ignorance of @enforce_keys and prevent invalid struct keys being passed.
If you want some code to do the two checks independently, you could add something like this:
use ExConstructor, check_enforce_keys: true, check_no_invalid_args: true
...
# pass @enforce_keys as 2nd param
def check_enforce_keys(args, enforce_keys) do
arg_keys =
args
|> Map.new
|> Map.keys
|> MapSet.new
required_keys = enforce_keys |> MapSet.new
if not MapSet.subset?(required_keys, arg_keys) do
raise(
ArgumentError,
"Requires keys #{required_keys |> Enum.to_list |> inspect} "
<> "but only #{arg_keys |> Enum.to_list |> inspect} "
<> "were given"
)
end
end
def check_no_invalid_args(args, module) do
arg_keys =
args
|> Map.new
|> Map.keys
|> MapSet.new
valid_keys =
module
|> struct([])
|> Map.from_struct
|> Map.keys
|> MapSet.new
if not MapSet.subset?(arg_keys, valid_keys) do
raise(
KeyError,
"Allowed keys are #{valid_keys |> Enum.to_list |> inspect} "
<> "but #{arg_keys |> Enum.to_list |> inspect} "
<> "were given"
)
end
end
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.