Add slack_reaction table to get message reactions in a channel. This is helpful to get reaction in messages and who added that reaction.

References:
[1] https://api.slack.com/methods/reactions.get

These errors are related to turbot/steampipe-postgres-fdw#3

select
  user_name,
  timestamp,
  channel ->> 'name' as channel,
  text
from
  slack_search
where
  query in('in:#steampipe from:nathan urgent after:3/12/2021', 'in:#steampipe from:kai urgent after:3/12/2021');

  Error: internal_error

 select
  user_name,
  timestamp,
  channel ->> 'name' as channel,
  text
from
  slack_search, my_domains
where
  query = name
Error: 'List' call requires an '=' qual for column: query

By default search results are paginated:
https://github.com/slack-go/slack/blob/93fe17cfad827ebb34316c683f29377c9be910e3/search.go#L9-L14

And in this implementation params are not allowed to be configured by users:
https://github.com/turbot/steampipe-plugin-slack/blob/main/slack/table_slack_search.go#L46

The problem with this is that we can't get more than 20 results in a single query. Maybe the pagination could be managed through "LIMIT" or by plugin config.

Describe the bug
Any select * from slack_ results in an error like:

Error: table 'slack_user' column 'workspace_domain' requires hydrate data from getCommonColumns, which failed with error missing_scope.
 (SQLSTATE HV000)

Steampipe version (steampipe -v)
v0.16.4

Plugin version (steampipe plugin list)
Slack v0.9.0

To reproduce
select * from slack_user

Expected behavior
The full list of users and their fields (although this is just based on the example provided)

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Describe the bug
When I run this query against my Slack workspace:

select
*
from
slack_user;

I do not get any users returning with populated data in the profile_fields column. However I know my own profile does have custom profile fields looking at my slack profile.
select * from slack_user where email = '[email protected]'
comes back profile_fields = []

Is there a bug here? Can you test on your end?

Steampipe version (steampipe -v)
steampipe version 0.13.3

Plugin version (steampipe plugin list)
Slack 0.3.0

To reproduce
Follow query above to a known user in your slack to have custom profile fields. Should not return an empty array

Expected behavior
Should be an array of key:values

Additional context
I did clear my cache, turn off cache, restarted my service, etc.

Is your feature request related to a problem? Please describe.
Add the pagination support in the table slack_user.

Currently we don't support pagination for listing out the users.
As per the API it supports pagination.

Describe the solution you'd like
Add pagination support for the table slack_user
API ref: https://pkg.go.dev/github.com/slack-go/[email protected]#Client.GetUsersContext.

This is a query against the Steampipe public Slack that gets pretty far, then triggers a rate limit and will not complete.

with days as (
  select
    to_char(day, 'YYYY-MM-DD') as day
  from
    generate_series(date('2021-04-15'), date('2021-11-16'), '1 day') as day
),
queries as (
  select
    'in:#steampipe on:' || d.day as query
  from
    days d
),
data as (
  select
    user_name,
    user_id,
    to_char(timestamp, 'YYYY-MM-DD') as day,
    text
  from
    slack_search
  where
    query in ( (select query from queries) )
  order by 
    day 
)
select 
  day,
  count(*) as messages
from
  data 
group by
  day

Here is my first attempt to use plugin.retryHydrate: https://github.com/turbot/steampipe-plugin-slack/compare/use-retry-hydrate#diff-dd4fe9f3414fd76a85ed3ab2bc2e8896acd02d8e0687afc4599bc1bfec51059f

It seems to work, in that with this change the query will eventually complete (Time: 8m50.043227805s)

Again it gets fairly far before the rate limit kicks in. Then, there is a lot of chatter between e.g. the first line in the log extract below that returns 59 messages for a day, and the next line that returns 1 message for the following day.

If the API is going to make the plugin wait 3 seconds, would it be best to just take that wait, for example in shouldRetryError, with time.Sleep(3 * time.Second)?

I've tried that, and it also eventually completes (Time: 5m53.883121854s).

It appears that the explicit wait finishes sooner, but I'm not sure about the time comparison because the point at which rate limiting kicks in may vary from run to run.

My questions:

• What is retryHydrate's backoff strategy? I thought it would increase the backoff when the plugin encounters resistance, but I don't think I'm seeing evidence of that here.

• Is taking an explicit wait the right strategy here?

2021-11-16T15:53:32.858-0800 [WARN]  steampipe-plugin-slack.plugin: [WARN]  slack_search: returning msgs: =59
2021-11-16T15:53:32.869-0800 [WARN]  steampipe-plugin-slack.plugin: [WARN]  slack_search: retrying="\"\""
2021-11-16T15:53:33.009-0800 [WARN]  steampipe-plugin-slack.plugin: [WARN]  slack_search: api.SearchContext="slack rate limit exceeded, retry after 3s"
2021-11-16T15:53:33.009-0800 [ERROR] steampipe-plugin-slack.plugin: [ERROR] slack_search.listSearches: retry_error="slack rate limit exceeded, retry after 3s"
2021-11-16T15:53:33.110-0800 [WARN]  steampipe-plugin-slack.plugin: [WARN]  slack_search: retrying="\"\""
2021-11-16T15:53:33.239-0800 [WARN]  steampipe-plugin-slack.plugin: [WARN]  slack_search: api.SearchContext="slack rate limit exceeded, retry after 3s"
2021-11-16T15:53:33.239-0800 [ERROR] steampipe-plugin-slack.plugin: [ERROR] slack_search.listSearches: retry_error="slack rate limit exceeded, retry after 3s"
2021-11-16T15:53:33.439-0800 [WARN]  steampipe-plugin-slack.plugin: [WARN]  slack_search: retrying="\"\""
2021-11-16T15:53:33.578-0800 [WARN]  steampipe-plugin-slack.plugin: [WARN]  slack_search: api.SearchContext="slack rate limit exceeded, retry after 3s"
2021-11-16T15:53:33.578-0800 [ERROR] steampipe-plugin-slack.plugin: [ERROR] slack_search.listSearches: retry_error="slack rate limit exceeded, retry after 3s"
2021-11-16T15:53:33.879-0800 [WARN]  steampipe-plugin-slack.plugin: [WARN]  slack_search: retrying="\"\""
2021-11-16T15:53:34.015-0800 [WARN]  steampipe-plugin-slack.plugin: [WARN]  slack_search: api.SearchContext="slack rate limit exceeded, retry after 3s"
2021-11-16T15:53:34.015-0800 [ERROR] steampipe-plugin-slack.plugin: [ERROR] slack_search.listSearches: retry_error="slack rate limit exceeded, retry after 3s"
2021-11-16T15:53:34.516-0800 [WARN]  steampipe-plugin-slack.plugin: [WARN]  slack_search: retrying="\"\""
2021-11-16T15:53:34.657-0800 [WARN]  steampipe-plugin-slack.plugin: [WARN]  slack_search: api.SearchContext="slack rate limit exceeded, retry after 3s"
2021-11-16T15:53:34.657-0800 [ERROR] steampipe-plugin-slack.plugin: [ERROR] slack_search.listSearches: retry_error="slack rate limit exceeded, retry after 3s"
2021-11-16T15:53:35.457-0800 [WARN]  steampipe-plugin-slack.plugin: [WARN]  slack_search: retrying="\"\""
2021-11-16T15:53:35.719-0800 [WARN]  steampipe-plugin-slack.plugin: [WARN]  slack_search: api.SearchContext=<nil>
2021-11-16T15:53:35.719-0800 [WARN]  steampipe-plugin-slack.plugin: [WARN]  slack_search: returning msgs: =1

PS: This branch also increases the gulp size (params.Count = 100) from the default which is evidently 20.

My use case is to audit single-channel users that are in no channels. At present, there's no way to look up which users in which channels. I can get lists of conversations and lists of single-channel users but no way, without conversations.members to get the intersection of those two tables.

References
https://api.slack.com/methods/conversations.members

Is your feature request related to a problem? Please describe.
When using multiple connections, it's difficult to tell which results are from which connection

Describe the solution you'd like
A common column would help differentiate rows

Describe alternatives you've considered
None

Additional context
Add any other context or screenshots about the feature request here.

References
Add any related links that will help us understand the resource, including vendor documentation, related GitHub issues, and Go SDK documentation.

Scopes configured as docs:

slack_access_log throws: Error: pq: rpc error: code = Unknown desc = paid_only - is this table not accessible on a free plan?

slack_search throws: Error: pq: rpc error: code = Internal desc = 'List' call requires an '=' qual for column: query

Would be happy to learn if I'm being stupid :)