lexmag / statix Goto Github PK

Hi I found this issue where having a sample rate of 0.001 will turns out equivalent to sending sample rate as 0.0. It was caused by the following segment of code:

There is no way to circumvent this limit from the user side. Should we perhaps raise the number of digits preserved?

Hi @lexmag ,

I was wondering if this project could use some additional help with support and maintenance. It looks like there have been no releases for a while and not much activity on the repo even with a handful of open PRs.

We use this library in a few projects at Bleacher Report and would be glad to help out where we can.

I would be in support of adding another maintainer or two, or moving the repo to an organization. Feel free to let me know how I can help or use this thread to discuss further.

Running in Elixir 1.14.5 and Erlang 26.0, I was experiencing the call to increment hanging, and then other calls in other processes erroring with lost value 1 due to port closure.

Trying to work out what was going on, I got down to:

iex(1)> init_conn = Statix.Conn.new(System.get_env("STATSD_HOST"), String.to_integer(System.get_env("STATSD_PORT")))
%Statix.Conn{sock: nil, header: [1, 31, 189, 172, 16, 138, 8, 0, 0, 0, 0]}
iex(2)> open_conn = Statix.Conn.open(init_conn)
%Statix.Conn{
  sock: #Port<0.27>,
  header: [1, 31, 189, 172, 16, 138, 8, 0, 0, 0, 0]
}
iex(3)> Statix.Conn.transmit(open_conn, :counter, "counter.testing", "1", [])
** (EXIT from #PID<0.857.0>) shell process exited with reason: :einval

This seems related to erlang/otp#7130 which won't be fixed.

After downgrading to Erlang 25, that sequence above works with an :ok result.

Is this project still maintained? Could the call to Port.command be changed / fixed?

Hi,

We've been happily using statix for our statsd needs (thanks!), and I'd like to play around with the distributions type, which datadog introduced a little while ago. To my understanding, it works by sending metrics with type d to the datadog statsd agent.

I looked at the code and currently the metric types are hard coded and rather difficult to extend without replacing both Conn and Packet...

Maybe something like the following could be supported?

defmodule MyApp.Statix do
  use Statix, custom_metrics_types: %{dd_distribution: "d"}

  def distribution(key, val \\ 1, options \\ []) when is_number(val) do
    Statix.transmit(current_conn(), :dd_distribution, key, val, options)
  end
end

Let me know if this sounds like a reasonable approach; I'd be happy to work on it and provide a pull request!

Do you think we can extend the library to include events and service-checks reporting?

https://docs.datadoghq.com/guides/dogstatsd/#events
https://docs.datadoghq.com/guides/dogstatsd/#service-checks

If you think we can go in that direction I will open a PR.

Hello!

First, thanks for this library :)

I was wondering how you would recommend using this library in test?

I've seen other libraries (such as this ruby one) may provide an in-memory backend that allows one to make assertions about the metrics being recorded. Do you have a similar system? Or is this not something you see value in testing? :)

Cheers,
Louis

There's at least one widely awaited change (#19) since the last release, how do you feel about pushing out a 1.2.0 release?

It looks like the UDP driver has changed in Erlang 21.3.8.4 which causes all Statix calls to fail. It appears that changing Port.command :gen_udp.send would fix this issue. Is there a reason to not use :gen_udp.send here? One benefit of :gen_udp.send is that the packet creation becomes a bit simpler.

21.3.8.3 - https://github.com/erlang/otp/blob/OTP-21.3.8.3/erts/preloaded/src/prim_inet.erl#L561
21.3.8.4 - https://github.com/erlang/otp/blob/OTP-21.3.8.4/erts/preloaded/src/prim_inet.erl#L568

Is there any way to know if the UDP packing is being sent?

Because I keep issuing:

Myapp.Statix.increment("page_view", 100)
:ok

and everything seems to work, but data dog is not showing anything.

Hi all.

I feel like I must be missing something obvious.

So if I have a runtime_config: true config for Statix (unsure if relevant here), and I do NOT explicitly start using MyApp.Statix.connect() (because it is disabled for whatever reason), the calls to Statix.increment etc are all seeming to fail complaining that I didn't set the __statix_header__.

"could not fetch application environment :\"Elixir.MyApp.Statix.__statix_header__\" for application :statix because configuration :\"Elixir.MyApp.Statix.__statix_header__\" was not set"

Is the expectation that I have to wrap all the calls in defensive logic to determine if Statix is running? Looking through the code, that would seem to be the case, as there doesn't appear to be logic in there to deal with this case. If so, I would like to propose being able to disable Statix for e.g. tests, and having the library itself handle the "send this metric only if running" logic rather than making app code do that.

Thanks!

Probably because of this line

For example, if I push a histogram with tags: ["errors:error1,error2"] , in Datadog it'll have two tags: errors:error1 and error2 .

Any way to escape the comma? Or maybe we should surround the tag with quotes if it has a comma? Or raise an exception ("a tag must not include a comma")?

I've been trying to test out statix, but I get an :eafnosupport error when trying to report metrics.

I have the a simple test module with default config:

defmodule Statixtest do
  use Statix
end

When I try to run it in iex -S mix I get the error:

iex(1)> Statixtest.connect()
:ok
iex(2)> Statixtest.increment "foo"
{:error, :eafnosupport}

I have a StatsD UDP port open on localhost:8125, same as default config.

I am running Erlang R19, Elixir 1.3.2, on Mac OS X.

Hey! Is it possible to use Statix with a unix socket?

Looks like the latest bump in OTP 21 from 21.3.8.3 to 21.3.8.4 completely broke statix for all my projects. Not really positive as to why. Upgrading to OTP 22 seems to have fixed the issue, but I thought people would want to know.

First off, thanks for this great library! It is one of a few libraries that I usually include in a new Elixir application by default.

In one of my higher throughput applications, we started noticing a high number of :busy_port notifications, which turned out to be coming from the Statix port.

I tried out #35 to see if perhaps we could reduce the number of notifications by connecting to the unix socket directly, and also tried #42, switching the Port.command/2 to :gen_udp.send/4, but I am still seeing many busy port notifications.

So I tried another strategy which I have used in this application to reduce pressure, which is to start multiple processes, and choose one randomly.

In my testing I was able to completely eliminate all :busy_port notifications using this strategy.

I am very interested to hear your thoughts on the matter and whether a PR would be accepted.

Here is my fork master...sneako:rand-pool.

Thanks again!

I am using influxdb and telegraf metrics for statsD
but I couldn't find the tags I am sending using statix, I am using Grafana to visualize the data.

Then I did little digging and found that
InfluxDB understands tags sent in this format

<tag_key>=<tag_value>,<tag_key>=<tag_value>

I changed the format according and used following syntax.

tag1 = "method=#{conn.method}"
tag2 = "handler=#{controller}_#{action}"
tag3 = "service=carbon"
tag4 = "env=#{Mix.env()}"
MyApp.Statix.increment(status_count, 1, tags: [tag1, tag2, tag3, tag4])

I am able to see the status_count measurement but tags are not available there in the grafana.

I've read the README and the source several times and I cannot figure out how to actually set runtime config.

I have this module:

defmodule Core.Statix do
  use Statix, runtime_config: true
end

Now, where do I actually put the config I want it to load at runtime?

I've tried setting it on application boot but I get this error in tests:

  def start(_type, _args) do
    configure_statix()
    Supervisor.start_link(children, strategy: :one_for_one)
  end

  defp configure_statix() do
    Application.put_env(:statix, :host, System.get_env("DATADOG_HOST") || "localhost")
    Application.put_env(:statix, :port, System.get_env("DATADOG_STATSD_PORT") || 8125)
  end

** (ArgumentError) could not fetch application environment :"Elixir.Core.Statix.__statix_header__" for application :statix because configuration :"Elixir.Core.Statix.__statix_header__" was not set

We often need to pre-configure statix at runtime before running the connect/0 function. Would it make sense to allow connect to accept a configuration? This would allow us to avoid mutating the application env. I can take a look at this if it's interesting to y'all.

Hi Folks,
I am able use statix for http metrics but I wanted to know if there is a way I can configure it to capture ecto metrics as well.

Thanks

Would you accept a PR for a global tags functionality, where you could provided tags that are sent with every metric, and set in configuration:

config :statix,
  host: "somehost.com",
  port: 8125,
  prefix: "our_app",
  tags: [Mix.env, "global_tag"]

This should provide better user experience and ergonomic to the library.

I will provide PR later.

Hey folks,

Maybe I'm doing something wrong, but if my statsd agent restarts all function calls end up raising as:

> Sensetra.Statsd.increment("elixir.foo", 1)
** (ArgumentError) argument error
    :erlang.port_command(Sensetra.Statsd, [<<1, 31, 189, 172, 31, 22, 97>>, "elixir.foo", 58, "1", 124, "c"])
    (statix) lib/statix/conn.ex:30: Statix.Conn.transmit/2

This seems undesirable, monitoring functions shouldn't nuke the app in the event that the port goes down.

Ideally there'd be some mechanism for trying to reconnect as well, but at a minimum it would seem we probably want to catch this error.

Are there plans to add TCP support in addition to UDP?

Just wondering since I haven't found a mention of it in the docs or issues.

I noticed that the Statsd port and ip configuration is evaluated at compile time only right? Please correct me if I'm wrong.

Would it be possible to change this so it can be defined during runtime? I understand that the header needs to be "saved" somewhere so that the Port.command will have it to build the full message.

What if we use Application env to keep the header there? So as soon as the connection is open we set the header for that connection?

Would that be too hacky?

I understand that this is a core feature from Statix (the ability to cache the packet header) but it's also very limiting the facet the Statsd host and port needs to be known during compilation :(

Hey all,

I'm now in the process of migrating some of our apps to use statix, and in the process of doing so I've found a really strange behavior. When calling use on the module I'm not passing the runtime_config option (which defaults to false), as I don't need to change the config at run-time. However, it looks like when using it this way, the connection is made at compile-time instead of run-time. For instance, when booting the application locally, if I don't have the statsd container running, my app won't compile with the following error: == Compilation error in file lib/motus/statix.ex == ** (MatchError) no match of right hand side value: {:error, :nxdomain}.

I've done some digging and placed some IO.inspects around, and it looks like the Statix.new_conn function is really being called at compile-time. However, since that code is inside a quote block, it should just return the AST instead of evaluating it, right? 🤔

Thanks for your help and for the library itself, it's really useful 🙌