[EPIC] Tasks for a new Top Level Apache Project about arrow-datafusion HOT 29 CLOSED

Its official -- DataFusion is now its own Top Level Project!

https://projects.apache.org/project.html?datafusion

I plan to file individual tickets for the tasks listed above later this morning

from arrow-datafusion.

My plan / hope is that we leave as much of the operations of DataFusion the same day to day as we transition to our own top level project. Once we have gotten ourselves running independently then we can discuss any potential changes

I think mailing lists are the first important thing to setup, so following https://github.com/apache/arrow/blob/main/.asf.yaml, I submitted requests via https://selfserve.apache.org for the following emails

• [email protected]
• [email protected]
• [email protected]
• [email protected]

In continuing with the current norms of the DataFusion community, I did not create a [email protected] mailing list, because according to the docs "The vast majority of communication occurs in the open on our github repository in the form of tickets, issues, discussions, and Pull Requests."

Thus I think having another potential list that people might accidentally send a request would be confusing.

from arrow-datafusion.

Thanks to @kou we have completed #10194 and the old doc links now redirect to datafusion.apache.org now

I tested a few links like
https://arrow.apache.org/datafusion/library-user-guide/working-with-exprs.html
https://arrow.apache.org/datafusion/user-guide/cli/index.html

from arrow-datafusion.

We will also need to make some updates to the release scripts and associated documentation (link to reporter URL, mailing list name, and so on)

from arrow-datafusion.

I created #10130 to make some documentation updates to change the name from Apache Arrow DataFusion to Apache DataFusion

from arrow-datafusion.

I filed an INFRA ticket to rename the repos.

https://issues.apache.org/jira/browse/INFRA-25726

from arrow-datafusion.

Ok, the DNS issue has been resolved.

We have the website up (needs some links fixed) https://datafusion.apache.org/

The mailing lists are working as well. For example: https://lists.apache.org/[email protected]

from arrow-datafusion.

https://datafusion.apache.org/ now has the website live 🚀 -- thanks @phillipleblanc

from arrow-datafusion.

We should rename slack and discord channels?

Update here is that @andygrove did so

from arrow-datafusion.

Note it is critical in my opinion that all existing links continue to work (as in the url https://github.com/apache/arrow-datafusion/issues/9691 will be redirected to the new repo). I believe this is the default behavior in Github when a repository is renamed but we should double check.

Yes, I just verified this behavior is the default. I created a repo at https://github.com/phillipleblanc/datafusion-rename-test - created an issue: https://github.com/phillipleblanc/datafusion-rename-test/issues/1 and renamed the repo to datafusion-rename-success - and now the above links redirect to the new repo name.

from arrow-datafusion.

I got an email aknowledging the creation of the email lists, but the domain name does not appear to be working. I filed https://issues.apache.org/jira/browse/INFRA-25727 to track

from arrow-datafusion.

Update .asf.yaml to point to to the new mailing list: #10189

I'm also taking a look at #10151

from arrow-datafusion.

DataFusion Top Level Project announcement is live: https://arrow.apache.org/blog/2024/05/07/datafusion-tlp/

from arrow-datafusion.

Thank you for double checking @phillipleblanc

from arrow-datafusion.

🎉🎉

from arrow-datafusion.

Regarding mailing lists, I think this may not be possible or hard to do, but just curious about if there are similar cases before. Is it possible to move previous threads related to DataFusion to new mailing lists?

from arrow-datafusion.

Regarding mailing lists, I think this may not be possible or hard to do, but just curious about if there are similar cases before. Is it possible to move previous threads related to DataFusion to new mailing lists?

I am not sure -- maybe we could look in the INFRA JIRA tickets or file a request

from arrow-datafusion.

I created #10133 and #10134 for logistics

I also filed #10135 to make a blog post -- I would be interested in anyone has opinions on the venue (should we post to the arrow blog or make the first post on a new datafusion specific blog)?

from arrow-datafusion.

@comphead pointed out to me that the commiters list of the new repo is not updated yet. I will fix that now #10154

from arrow-datafusion.

I also filed #10155, #10156, and #10157 to track the board reports (and build some institutional knowledge)

from arrow-datafusion.

We are sill having DNS trouble -- filed https://issues.apache.org/jira/browse/INFRA-25731

from arrow-datafusion.

this link currently still works and doesn't redirect -- will it redirect to the new one at some point? https://arrow.apache.org/datafusion/

(I know this is very new and in flight, appreciate the work here!)

from arrow-datafusion.

We should rename slack and discord channels?

from arrow-datafusion.

this link currently still works and doesn't redirect -- will it redirect to the new one at some point? https://arrow.apache.org/datafusion/

@lostmygithubaccount (😆 ) Yes absolutely -- here is a PR to do that apache/arrow-site#502

from arrow-datafusion.

Update: @tisonkun has made a DOAP file 😄 -- #10233

I have created a proposed page with governance information: #10238

from arrow-datafusion.

I filed a few more doc tweaks #10284 and #10285

I think all that is left for this epic is to write a blog post (#10135) and we can close it down

from arrow-datafusion.

Actually, we also owe the ASF board a report each month for the first 3 months. I'll begin coordinating the first one shortly (tracked via #10281)

from arrow-datafusion.

I have created a draft blog post on the arrow site for announcing the new top level project: apache/arrow-site#512

from arrow-datafusion.

Also, we got a suggestion to make an official ASF press releas #10403

from arrow-datafusion.