//Build 2016 Recap

[jamessdixon]

I went to //Build 2016 and tried to go to the most sessions and talk with the most people about data science.  Microsoft is very interested in this: - though not from a F# point of view.  This is the products/areas they are pushing

1) Azure.  AzureML and Cognative services.  They want to lock in your organization's data and ML models onto Azure

2) R Server and Sql Server.  They want you to write R in Visual Studio and deploy to R Server and then communicate to Sql Server to execute your models.  They store the models as a byte array in a sql server table and call them from the R code embedded into a stored proc.

3) Everything about their IoT story is about reading and reporting real-time data and using Power BI. No discussion about predictive modeling.  There were a TON of sessions on this stuff.  They have a language about USql to basically do custom filtering and transformations on their IoT pipeline

I talked with some really cool MSFT people on these projects.  All are great but no one is really interested in F# AFAIK.  It is worth you time to follow these people on Twitter

@danielleodean

@seesharp

@John_Lam

@revodavid

@harryshum

@jennifermarsman

Here are some of the sessions that are worth your time:

https://channel9.msdn.com/Events/Build/2016/B884

https://channel9.msdn.com/Events/Build/2016/B805 

https://channel9.msdn.com/Events/Build/2016/B837

https://channel9.msdn.com/Events/Build/2016/B878

[Steffen Forkmann]

Any ideas why they are not interested?

--
You received this message because you are subscribed to the Google Groups "fsharp-data-science" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fsharp-data-sci...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jamie Dixon]

I did not ask that specific question, but based on other discussions about F# adoption, it gets down to numbers -> they think they can drive adoption with their dev based but staying in C# - heck they even built this abomination called USql so that C# devs would not fear the change.

[Steffen Forkmann]

Ok thanks.

The funny thing is that C# world and data science world are pretty much disjunct. In my opinion it's a bad strategy and will not bring new people to the platform. But what do I know.

[Mathias Brandewinder]

Thanks for the recap Jamie!

I have to say I am still a bit puzzled by the whole R + SQL thing, this sounds like a deeply unholy idea.

From this list, the IoT / real time part sounds to me like a place where F# should shine. I wonder what a proof of concept / example would look like - and also if FsStorm might be a good fit there.

Maybe it's just me, but I get the sense that MSFT has a lot to bring to the table in that space, but they are playing catch-up, looking at popular topics and try to bolt popular tools onto their offering. Except for the DataLake direction, I have a bit of a hard time understanding the overall story.

[Jamie Dixon]

I think we can help drive the discussion by:

1)      Talking to the MS Data Science Group:  http://msdsug.microsoft.com/ and offering to write articles for their newsletter

2)      Talking to Harry Shum (Data Science)

3)      Talking to Sam George (IoT)

4)      Talking to someone other than Jim McCaffrey at MSDN about doing a series of Data Science articles.  Combine what MSFT is interested in with F#

5)      Talking to Channel9 and Dot Net Rocks about F# data science success stores and not “it’s a great language” session

 

Doing High Profile Kaggle and MSFT competitions in F# and then socializing it.  Getting on Channel 9 to talk about how we are the highest ever .NET team at Kaggle or winning the MSFT EKG completion will go a long way, I think…

MSFT has no story – they have millions of developers and paying customers that don’t know better.  Just need to educate with success. J

 

From: fsharp-da...@googlegroups.com [mailto:fsharp-da...@googlegroups.com] On Behalf Of Mathias Brandewinder
Sent: Saturday, April 2, 2016 2:22 PM
To: fsharp-data-science <fsharp-da...@googlegroups.com>
Subject: Re: //Build 2016 Recap

 

Thanks for the recap Jamie!

--

[Mathias Brandewinder]

I largely agree. It also makes me a bit sad. As an example of things that make me sad, Spark attracted quite a significant number of new people to Scala (who were clearly not FP fanatics), and I think Microsoft had everything to be the leader in that space - DryadLINQ, F#, and completely missed the boat... Instead, today, they are working on making it possible to run Spark on Azure :(

</rant>

[Seth Juarez]

I'm happy to help wherever I can


--
Seth Juarez

[Daniel Egloff]

Dear all

I also attended the build conference and I can confirm the interest of Microsoft in data science. What is very interesting is their approach: they try to deliver data science services as a commodity. That means they try to shield the user from the complexity of ML and data analysis. This has pros and cons. It is certainly good for mainstream usage, which has a lot of business potential. 

F# might fit more in the area of "explorative data analysis". There the current offering is very week, unless you start writing your own data analysis stack on top of Spark and related tools. 

In this aspect I find the recent Prajna library of Microsoft research a very interesting alternative. To be realistic however, the competitor Spark is almost a defacto standard and together with Scala gives a rather compelling data analysis / ML stack. 

So the question is what should be done so that F# does not miss the boat in data science. Unfortunately, currently it looks like that is going to happen. 

[Jamie Dixon]

And getting more people to contribute to Accord and numl

J

 

 

From: Seth Juarez [mailto:m...@sethjuarez.com]
Sent: Saturday, April 2, 2016 2:53 PM
To: Jamie Dixon <james...@gmail.com>
Cc: 'Mathias Brandewinder' <mathias.br...@gmail.com>; 'fsharp-data-science' <fsharp-da...@googlegroups.com>
Subject: RE: //Build 2016 Recap

 

I'm happy to help wherever I can


--
Seth Juarez

On Apr 2, 2016, 11:33 AM, Jamie Dixon wrote:

I think we can help drive the discussion by:

1.      Talking to the MS Data Science Group:  http://msdsug.microsoft.com/ and offering to write articles for their newsletter

2.      Talking to Harry Shum (Data Science)

3.      Talking to Sam George (IoT)

4.      Talking to someone other than Jim McCaffrey at MSDN about doing a series of Data Science articles.  Combine what MSFT is interested in with F#

5.      Talking to Channel9 and Dot Net Rocks about F# data science success stores and not “it’s a great language” session

 

Doing High Profile Kaggle and MSFT competitions in F# and then socializing it.  Getting on Channel 9 to talk about how we are the highest ever .NET team at Kaggle or winning the MSFT EKG completion will go a long way, I think…

MSFT has no story – they have millions of developers and paying customers that don’t know better.  Just need to educate with success. J

 

From: fsharp-da...@googlegroups.com [mailto:fsharp-da...@googlegroups.com] On Behalf Of Mathias Brandewinder
Sent: Saturday, April 2, 2016 2:22 PM
To: fsharp-data-science <fsharp-da...@googlegroups.com>
Subject: Re: //Build 2016 Recap

 

Thanks for the recap Jamie!

 

I have to say I am still a bit puzzled by the whole R + SQL thing, this sounds like a deeply unholy idea.

From this list, the IoT / real time part sounds to me like a place where F# should shine. I wonder what a proof of concept / example would look like - and also if FsStorm might be a good fit there.

 

Maybe it's just me, but I get the sense that MSFT has a lot to bring to the table in that space, but they are playing catch-up, looking at popular topics and try to bolt popular tools onto their offering. Except for the DataLake direction, I have a bit of a hard time understanding the overall story.

--
You received this message because you are subscribed to the Google Groups "fsharp-data-science" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fsharp-data-sci...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Steffen Forkmann]

Tbh given MS history on this it's pretty useless to discuss any further. We had this discussion for years.

MS is putting lots of people and money on some super strange things that no data scientist will ever use. Criticizing this openly will bring you lot of negative vibes. Scala + JVM ecosystem is so far ahead and there is no way to catch up with C# and SQL. I really wonder who inside this company thinks that that's a good strategy.

[Matthew Smith]

Hi Guys,

 

As a Microsoftie insider (although all this is just my opinion) involved in the whole data science movement in the company, a lot of the obvious developments are catering to the current demand to show the value of having data science capability within the platform. The largest group of data scientists (numbering hundreds now), under the azure machine learning development group, are largely focussed on addressing customer problems and R, SQL, .NET are used for that interfacing in many circumstances. I totally agree F# is a brilliant language to pick for data science (one of the best) but the approach I see is not to push a language or approach but to enable us and customers to solve real current business problems. A problem for F# is that it’s not the first language of choice of the data scientists being brought in to solve the problems and the difficulty in solving the problems is not about having a good language, but finding an effective solution.

 

To me, the best way to drive F# adoption for data science within microsoft is to get it used in real customer engagements. It’d be cool if we could have an F# module in azure machine learning, to go alongside the R and Python ones. I’d personally use it. I wonder if someone could take some of the template solutions in the Cortana intelligence gallery https://gallery.cortanaintelligence.com/?r=legacy&fromlegacydomain=1 and show where and how F# would help (that’d be a good blog). That might tempt more people over.

 

Just some musings
Matthew

______________________________________________

Matthew Smith

Senior Solution Architect, Microsoft

+44(0)7980659917, LinkedIn, @JungleTeuch, Scholar

 

From: fsharp-da...@googlegroups.com [mailto:fsharp-da...@googlegroups.com] On Behalf Of Jamie Dixon
Sent: 02 April 2016 19:34
To: 'Mathias Brandewinder' <mathias.br...@gmail.com>; 'fsharp-data-science' <fsharp-da...@googlegroups.com>
Subject: RE: //Build 2016 Recap

 

I think we can help drive the discussion by:

[Steven Taylor]

Hi,

(as an outsider trying to work with)

Essentially, the magic is within the language itself, so don't get downhearted.

The moves appear mostly to be safe bets.  Slightly illogical to a data science perspective, but consistent with an engineering mindset and / or theme.  Possibly on the whole people are talking about it more than they are doing it.  You'd have to assume that MS have the research to make reasonably informed decisions on this.  

My question is really, given the will, how could a community like F# work with Microsoft's internal efforts.  For example, a few years ago I thoroughly went through the Python / Visual Studio integration and concluded that it was a nice piece of work.  The interfaces were clean and simple, and the approach was well thought out.

The python stuff was quietly released, but clearly part of a much larger push.  I *could* have made more noise about this within the F# community.  I did make some... anyway... for arguments sake let's assume that we all missed it.

(fast forward)

By the time we find out about such initiatives, it's often too late to get on board and synergise with whatever PR push is going on (the plan, build, launch cycle).  

I'd like to encourage Microsoft to do better community outreach on their initiatives.  The will in the F# community is there, but we are quite out of sync.

Going back to the python initiative, how could we have worked with MS while the ground work was going on?  Could there be some resources put aside to deal with random (tending towards structured) enquiries from the likes of me?

So if the will *is* there, and the the altruistic mindset too, ho do we work together?

A lot of the ground work for these frameworks is not rocket science.  I'll still take my hat off to the way the python integration was put together... and probably there are pieces that are just plain brilliant... but we are all in IT... we know pretty well how this stuff works.  Deciphering various corporate mindsets is tricky.

cheers,

-Steven

[Jamie Dixon]

Hey Matt:

Color me stupid but I didn't think AzureML allowed models to be added.  When the F# group talked to the Azure team 2 years ago at the MVP summit, we were told that it wasn't possible.  Looks like I need to reacquaint myself with it in that manner. 

In any event, I am excited in the direction that MSFT is going.  I think intelligent apps are a great way to start merging business application development and with data science.  Indeed, I do wonder if we will get to the point where software engineer and data scientist are indistinguishable.

[Mathias Brandewinder]

By the time we find out about such initiatives, it's often too late to get on board and synergise with whatever PR push is going on (the plan, build, launch cycle).  
I'd like to encourage Microsoft to do better community outreach on their initiatives.  The will in the F# community is there, but we are quite out of sync.

+1 on this. 

And sorry for the rant earlier, for some reason I got a bit worked up :)

--

Mathias Brandewinder

@brandewinder / http://brandewinder.com/

Check out my book, "Machine Learning Projects for .NET Developers"

[Ryan Riley]

Based on conversations I've had with my science team coworkers, the biggest need involves libraries of algorithms. We currently use Python with pandas and I think numpy and scipy, as well as Matlab. We tried getting everyone on F# earlier, but the libraries were just not there, from what I understand. What would it take to rectify that problem? Fix it with F#, and people will make the move. It is a big investment, to be sure. I suppose type providers to Python (or some specific libraries) could serve as a band-aid. However, with Python taking a hard stop on 2.7 and trying to finally force people onto 3, we may have a window of opportunity to move that whole community to F#. That said, my knowledge about the state of Python is based on second and third hand knowledge. Others would likely know better than me whether this is realistic. I would be happy to contribute to this effort, and I think others at Tachyus would, as well.

Sent from my iPhone

[Isaac Abraham]

Just my two cents.

 

I’ve had some dealings with the Azure data teams, particularly around the data lake / USQL side of things, both (a) to see if it’s possible to achieve some integration with F# and (b) to provide some feedback on where I felt USQL was difficult to use, particularly the “getting started” story for a C# developer, which I feel is massively overcomplicated (especially if you compare it to MBrace on Azure which is a doddle). As has been mentioned elsewhere, I simply don’t think that this is of interest and / or a priority to the teams – one of the goals is clearly to bring data science + ML to the masses so F# doesn’t really fit in there.

 

Spark on HDI seemed (at least when I last checked) to be at a relatively early stage – does it run on Linux yet? The Windows HDI distro when I last checked only allowed interaction through Jupiter really, there was no remote SSH etc. because it was all on Windows. Hopefully it’s gotten further than that now - Brisk had (has?) Spark on Linux on Azure 18 months ago…

 

Matt: You make a good point regarding data science languages etc. and I agree – this isn’t going to be about market making for F#’s sake. I think what sometimes grates though is when MS pushes C# ahead of F# as the .NET companion language for data science when F# is in virtually every case a better fit. I understand why that is, but it’s still a little surprising. If you look at MBrace you can see a really good (IMHO) integrated story for .NET and distributed compute / data.

 

Cheers

 

Isaac

 

From: fsharp-da...@googlegroups.com [mailto:fsharp-da...@googlegroups.com] On Behalf Of Jamie Dixon
Sent: 02 April 2016 20:17
To: 'Steffen Forkmann' <sfor...@gmail.com>
Cc: 'fsharp-data-science' <fsharp-da...@googlegroups.com>
Subject: RE: //Build 2016 Recap

 

I did not ask that specific question, but based on other discussions about F# adoption, it gets down to numbers -> they think they can drive adoption with their dev based but staying in C# - heck they even built this abomination called USql so that C# devs would not fear the change.

[Riccardo Terrell]

This is annoing and unfortunately repetitive.

I worked 3 years for MSFT, my duty was developing POC for other teams to satisfies some requirements. 

it was awesome because I was free to use any technology that I wanted, and I probably have used F# for 80% of the times. In these cases, at the end of each meeting and demos I got always the same comment:

“This is great, but it can be done in C#?”

My point is that the slow adoption of F# is due to a cultural and generational issue. There are still too many “old mind-set engineers” that are in charge of decisions, and they rather staying in their comfort-zone than move to a greener yard. 

Today is Data Science and ML, tomorrow will be something else. 

I am 100% agree with Jamie, we should just keep what we are doing in the community, talking and involving different “neighborhoods”,  be more present in the MSDN magazine to reach brother audience will defiantly help.

It is by educating young generation today that we will not encounter in the similar future situations.

Ciao

Riccardo 

[Matthew Smith]

Hi Steven,

It’s a damn good question, about how MS can be best working with the F# community while the DS developments are going on. Obviously a good question to ask senior management (I’d add Joseph Sirosh to the contacts list) and I can’t provide an authoritative answer. Sorting out a set of knock out examples to help make the case would be good (e.g. competitive advantage of using F# in big data analytics in the financial sector?)

Best,

M

______________________________________________

Matthew Smith

Senior Solution Architect, Microsoft

+44(0)7980659917, LinkedIn, @JungleTeuch, Scholar

 

[Matthew Smith]

Hi Jamie,

Yes indeed it does. Use an R code module. … basically a container for R code… with some restrictions on data input and output. I had the nice experience last month of taking some R code I’d been working with a customer on last year and embedding it in AML and setting up a web service within 10 mins.. and I’d never used AML before. V. liberating.

 

I think there will be a smoother spectrum than ever before between software engineer and data scientist… however the extreme ends will remain very very different. Like political parties.

 

Best,

M

 

______________________________________________

Matthew Smith

Senior Solution Architect, Microsoft

+44(0)7980659917, LinkedIn, @JungleTeuch, Scholar

 

From: Jamie Dixon [mailto:james...@gmail.com]

Sent: 03 April 2016 00:28
To: Matthew Smith <Matthe...@microsoft.com>

[Arturo Hernandez]

Matthew,

It is unfortunate that neither C# or F# are languages of choice for data science. But I do think that,  to the degree that decisions are not be mutually exclusive, MSFT should support both. Plus it's silly not to. If you happen to have an offering that includes a language that has more academic appeal, why wouldn't MSFT leverage that. Why reject existing and potential customers?

This sounds more like MSFT already has really great C# expertise in house. Regardless of the what the data science world may think or want. You are not going after the Spreadsheet market, you are going after leading edge. I think you need to take more risks in this case.

Good luck