Building Intelligent Apps and Agents with the Computer Vision API, Custom Vision Service, LUIS, Bing Search, and Azure Search

Welcome to the Learn AI Bootcamp for Emerging AI Developers. We will focus on hands-on activities that develop proficiency in Azure Cognitive Services including Computer Vision, Custom Vision, LUIS, and Bing Search. You will also learn to leverage other AI-oriented services such as Azure Search and Azure Bot Services. These labs assume an introductory to intermediate knowledge of these services, and, if this is not the case, then you should spend the time working through the prerequisites.

!!Important note 09/28/2018!!: The Microsoft Bot Builder SDK V4 went GA at Ignite this week. The bot labs have been updated to SDK V4, but the SDK V3 labs are located in the resources folder for the bot labs (not being maintained). Additionally, we have moved the logging and testing labs to supplementary exercises (not being maintained at the moment), and the bootcamp will focus on the addition of the Bing Search APIs to bots and other applications.
If you are an instructor redelivering this course and have questions, please email [email protected].

Most challenges observed by customers in these realms are in stitching multiple services together. As such, where possible, we have tried to place key concepts in the context of a broader example.

At the end of this workshop, you should be able to:

• Understand how to configure your apps to call Cognitive Services
• Build an application that calls various Cognitive Services APIs (specifically Computer Vision)
• Effectively leverage the custom vision service to create image classification and object detection services that can then be leveraged by an application
• Understand how to implement Azure Search features to provide a positive search experience inside applications
• Configure an Azure Search service to extend your data to enable full-text, language-aware search
• Build, train, and publish a LUIS model to help your bot communicate effectively
• Build and publish an intelligent bot using Microsoft Bot Framework that leverages LUIS and Azure Search
• Add Cognitive Services (specifically the Bing Search APIs) to your existing applications

This workshop is meant for an AI Developer on Azure. Since this is only a short workshop, there are certain things you need before you arrive.

Firstly, you should have some previous exposure to Visual Studio. We will be using it for everything we are building in the workshop, so you should be familiar with how to use it to create applications. Additionally, this is not a class where we teach you how to code or develop applications. We assume you have some familiarity with C# (intermediate level - you can learn here and here), but you do not know how to implement solutions with Cognitive Services.

Secondly, you should have some experience developing bots with Microsoft's Bot Framework. We won't spend a lot of time discussing how to design them or how dialogs work. If you are not familiar with the Bot Framework, you should complete this tutorial prior to attending the workshop.

Thirdly, you should have experience with the portal and be able to create resources (and spend money) on Azure. We will not be providing Azure passes for this workshop.

Finally, before arriving at the workshop, we expect you to have completed 1_Setup along with configuring the following for Custom Vision:

• Training API Key: The training API key allows you to create, manage and train Custom Vision project programmatically.
  • You can obtain a key by creating a new project at https://customvision.ai and then clicking on the “setting” gear in the top right.

Please note: This is a rough agenda, and the schedule is subject to change pending class activities, breaks, and interactions.

• Day 1: Cognitive Services
  • 8-8:30 (optional): Setup assistance
  • 8:30-9:30: Introduction to Cognitive Services
  • 9:30-10:30: Lab 1.1: Simplifying Cognitive Services App Development using Portable Class Libraries
  • 10:30-11:30: Lab 1.2: Developing Intelligent Applications with Azure Search
  • 11:30-12:00: Lab 1.3: Developing Intelligent Applications with LUIS
  • 12-1: Lunch
  • 1-1:30: Introduction and Context for Bots
  • 1:30-3:00: Lab 2.1: Building Intelligent Bots
  • 3-4: Lab 2.2: Log Chat Conversations in your Bot
  • 4-5: Lab 2.3: Testing your Bot

The LearnAI team has provided some extra labs you may be interested in:

Here are some related courses from the LearnAI team:

• LearnAI: Intelligent Agents: Design and Architecture
• LearnAI: Building Enterprise Cognitive Search Solutions

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.