F# Big Data Stacks page about fsharp.org HOT 12 CLOSED

Maybe "Big Data" and "Big Compute" could be merged

Some links

• http://blogs.msdn.com/b/carlnol/archive/2011/12/16/hadoop-streaming-and-f-mapreduce.aspx
• http://atbrox.com/2011/02/07/an-example-of-using-f-and-c-netmono-with-amazons-elastic-mapreduce-hadoop/
• http://visualstudiomagazine.com/articles/2012/10/16/linqing-dotnet-and-hadoop.aspx
• http://hadoopsdk.codeplex.com/
• http://blogs.msdn.com/b/cloudnumerics/archive/2012/01/16/cloud-numerics-example-distributed-numerics-on-azure-with-f.aspx

from fsharp.org.

@quasilord I like that suggestion. I added the links you mentioned that I didn't have already as well as some verbiage talking about "big compute" and a list of distributed messaging packages I am familiar with. I think this "feels" better and will make for a more substantial page. Any other ideas for material that should be included?

@tpetricek @dsyme either of you have a vision for this page along that isn't included here / different?

Comments here or direct edits to the gdoc are welcome.

from fsharp.org.

I'd recommend considerably less verbiage. This is not the right place to explain the (very large and rapidly evolving) universe of big data and big compute nor the techno-social trends around it. Just jump straight in with a condensed, snappy version of para #2 and #4 and then give links to resources. Assume the user is looking for one of "EC2", "Azure", "Hadoop", "ZeroMQ", "No-SQL", "Cassandra", "SQL", "MPI" etc.

Many people's data needs will actually be satisfid by SQL programming (people don't need Hadoop quite as much as they think), and that angle should be covered.

In this vein the title should probably be something like "Data, Compute, Cloud and Messaging Stacks for F#" without the "Big".

Type providers should be mentioned ("F# is the best language for Big Metadata") with specific mention of the SQL and OData type providers in Visual Studio and the FSharpx community type providers (e.g. for JSON, Xml etc.)

Trawling Nuget for related packages would be a good thing.

I suspect there should be specific sections for "Using F# on Azure" and "Using F# on EC2 and Rackspace". The Visual F# team might like to give input for the former. For example, that would cover

• Using F# on Azure VMs (both Linux and Windows)
• Azure AppFrabric programming wih F#
• Using SQL Azure with F#
• Using HDInsight (Azure Hadoop) with F#

EC2 would probably mostly just say the obvious - that the open source release of F# can be installed and used on EC2 VMs.

from fsharp.org.

There's a nice sample of using MongoDB here: https://gist.github.com/4394128

Cheers!
don

from fsharp.org.

@dsyme I like these suggestions a lot and this will help focus the content a bit. I'll take a look at the gist as well and, kids permitting, hopefully update the content this evening. :)

from fsharp.org.

Hi Jason, have you been able to update this as yet? The Azure guys at Microsoft want to add links to some tutorial material too, so it would be good to integrate what you have (in any state)

from fsharp.org.

Hi Don,

What's in the Google doc is what I have. I had a much-delayed beta release
to get out at work. Fortunately that is done so I will have time to get
back to this page.

Content from the Azure folks would be great. They are welcome to edit the
doc directly, email me content, or move the content to an Office doc and
share it back if that easier.

I should have time to clean things up this weekend if they would rather
have more structure in place first.

Jason
On Feb 1, 2013 7:05 AM, "Don Syme" [email protected] wrote:

Hi Jason, have you been able to update this as yet? The Azure guys at
Microsoft want to add links to some tutorial material too, so it would be
good to integrate what you have (in any state)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/42#issuecomment-12993228.

from fsharp.org.

Ok, I finally have gotten time to add the Windows Azure and Amazon Web Services sections. Comments on the content, particularly the scope and level of detail, are welcome.

@dsyme , you mentioned that the Azure guys are interested in adding links to additional material. They are welcome to edit the doc directly or they can email me the material (jasonmccampbell @ gmail) and I'll add it.

from fsharp.org.

Hi Jason, I can take a pass over the material, e.g. for Azure, Hadoop, and also for general wording. Can you send a pull request to submit what you have? Thanks!

from fsharp.org.

Hi Don, will do. I realized a couple of days ago that I was writing to the
wrong audience and cleaned things up a bit. I should have time to add some
additional content and submit a PR this weekend.

On Thu, Mar 14, 2013 at 9:07 AM, Don Syme [email protected] wrote:

Hi Jason, I can take a pass over the material, e.g. for Azure, Hadoop, and
also for general wording. Can you send a pull request to submit what you
have? Thanks!

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/42#issuecomment-14903878
.

from fsharp.org.

Move to PR: #53

from fsharp.org.

Closed (though some further work is needed per above discussion)

from fsharp.org.