Organizations are leveraging social media as a critical component of their marketing strategy. Social media provides a platform to connect with their customers, increase awareness about a brand, and improving leads and sales.
Social Media analytical tools can help analyze user behavior, market trends, and perception of any specific product and marketing campaign. Twitter is one of the social media platforms widely used by an organization for marketing.
In this article, we will perform sentiment analysis on live twitter feed via stream analytics and Azure cognitive Services.
Process Flow
As shown in the figure, a twitter application will send data to the event hub, azure stream analytics will push data to blob storage. A Blog Trigger Function would read the content on File upload call Azure Cognitive Service – Text Analytics API. The output from the API will be saved into the output folder of the container by the function.
Pre-requisites
- A Twitter account.
- The TwitterClientCore application, which reads the Twitter feed. To get this application, download TwitterClientCore. Download here: https://github.com/Azure/azure-stream-analytics/tree/master/DataGenerators/TwitterClientCore
- Install the .NET Core CLI version 2.1.0.
- An Azure storage account.
Create an Event Hub Namespace and Event Hub
- Log in to azure portal.
- Go to Event Hub. Click on Add.
- Enter details and click on Create.
- Go to resource, Click on + Event Hub.
- Provide Name and click on Create.
Grant Access to Event Hub
Grant access to send data to Event Hub. A policy needs to be created to allow access.
- Under Entities, click on Event Hub and select Event Hub created in the previous step.
- Under Settings, select Shared access policies. Click on Add.
- Provide policy name, select Manage, and click on Create.
- Select the policy created in the previous step.
- Copy Connection String and save it.
Create Twitter Application
- Browse to https://developer.twitter.com/en/apps . Click on Create App.
- Create a developer account if prompted.
- Provide app name and click on Get Keys.
- Copy API Key and API Secret Key and Save it.
- Go to App Dashboard and Generate Access Tokens.
- Copy the Access token and Access token secret and save it.
Configure Twitter Application
- As mentioned in the pre-requisites, download the Twitter Application.
- Open the App.config and downloaded as mentioned in the pre-requisites. Update the following settings:
- <
oauth_consumer_key> : API key.
- <
oauth_consumer_secret> : API secret key.
- <
oauth_token>: Access token.
- <
oauth_token_secret> : Access token secret.
- <
EventHubNameConnectionString> : Connection string.
- <
EventHubName> : Event hub name.
- <
- Open Command Prompt, navigate to the application directory, and execute the below commands:
dotnet build
dotnet run- The application starts sending data to the event hub.
Create Stream Analytics Job
- Go to Stream Analytics Job and Click on Add.
- Provide details and click on Create.
Create Stream Analytics Input
- Under Job Topology, click on Input.
- Click on Add stream input and select Event Hub.
- Provide Input alias and other details. Click on Create.
Create Stream Analytics Output
- Under Job Topology, click on Output.
- Click on Add and select Blob Storage/Data Lake Storage.
- Configure the storage and click Save.
Create Query
- Click on Query.
- Type Below sample query and Click on Save Query.
SELECT System.Timestamp as Time, text
FROM TwitterStreamInput
Input
WHERE text LIKE '%Azure%'- Start the Stream Analytics Job.
Provision Azure Cognitive Service – Text Analytics
Provision Text Analytics Service to invoke client APIs and analyze the contents of file
- Log in to the azure portal.
- Search for Text Analytics and Click on Text Analytics under Marketplace.
- Provide Name and Click on Create.
- Go to Resource > Keys and Endpoint. Copy Values of Key and Endpoint. These values would be configured for calling the APIs.
Create Azure Function
Create an Azure Function to deploy blob triggers. This trigger will get invoked when a file is uploaded to blob storage.
- Open Visual Studio.
- Click on Create a New Project.
- Search for Azure Functions Template.
- Select the template and click on Next.
- Provide Project Name and click on Create.
- Select “Blob Trigger”. Configure the storage account created in previous steps. Provide the Connection String Name and the container details in the path. Click on Create.
Add Cognitive Service Nuget Package
- Right Click on Project > Manage Nuget Packages.
- Search for “Microsoft.Azure.CognitiveServices.Language.TextAnalytics”. Select and Click on Install.
Create Azure Cognitive Services Client
- Right Click on Project > Add > New Item…
- Select “Class”, provide name and click Add.
- Update the SentimentAnalysis.Cs with the below Code.
In this code we have created a static Run method to create a Text Analytics Client and invoke SentimentAnalysis() method.
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Microsoft.Azure.CognitiveServices.Language.TextAnalytics;
using Microsoft.Azure.CognitiveServices.Language.TextAnalytics.Models;
namespace Microsoft.AzureCognitiveServices.TextAnalytics
{
public static class SentimentAnalysis
{
public static string Run(string endpoint, string key, string fileContent, string language = "en")
{
try
{
var credentials = new ApiKeyServiceClientCredentials(key);
var client = new TextAnalyticsClient(credentials)
{
Endpoint = endpoint
};
// var ent = client;
var result = client.Sentiment(fileContent, language);
string sentimentOutput = result.Score?.ToString();
//Convert Sentiment Score to Sentiment Type
double sentimentOutput_numeric = System.Convert.ToDouble(sentimentOutput);
string sentimentType = "Neutral";
if (sentimentOutput_numeric < 0.5)
sentimentType = "Negative";
else
if (sentimentOutput_numeric > 0.5)
sentimentType = "Positive";
//return sentimentOutput;
//Return Sentiment Score and Type
string sentiment = "Sentiment Score is " + sentimentOutput + "\nSentiment Type is " + sentimentType;
return sentiment;
//return sentimentOutput;
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
return "Error in finding the Sentiment";
}
}
}
}
Update the Azure Function
- Right Click on the Function and rename it.
- Update the below code in the Function.
The below code triggers on any file upload to the Input folder in the container. Reads the content of the file and calls the Run method on the SentimentAnalysis class. The output of the method is saved in the output folder of the container.
using System;
using System.IO;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Configuration;
namespace Microsoft.AzureCognitiveServices.TextAnalytics
{
public static class SentimentAnalysisFn
{
[FunctionName("SentimentAnalysisFn")]
public static void Run([BlobTrigger("%sentimentinput%/{name}", Connection = "textanayticsconn")]Stream myBlob,
[Blob("%sentimentoutput%/{name}", FileAccess.Write)] Stream validationOutput, string name, ILogger log,
ExecutionContext context)
{
var config = new ConfigurationBuilder()
.SetBasePath(context.FunctionAppDirectory)
.AddJsonFile("local.settings.json", optional: true, reloadOnChange: true)
.AddEnvironmentVariables()
.Build();
var Endpoint = config["endpoint"];
var SubscriptionKey = config["subscriptionkey"];
log.LogInformation($"Text Analytics - Sentiment Analysis blob\n Name:{name} \n Size: {myBlob.Length} Bytes");
StreamReader reader = new StreamReader(myBlob);
string fileContent = reader.ReadToEnd();
string sentimentOutput = SentimentAnalysis.Run(Endpoint, SubscriptionKey, fileContent);
using (var sw = new StreamWriter(validationOutput))
{
try
{
sw.Write(sentimentOutput);
sw.Flush();
}
catch (Exception ex)
{
log.LogInformation(ex.Message);
}
}
}
}
}
The Market has many analytical tools including one of Twitter’s own out of box analytical tools that can be used for analyzing Twitter data and trends.
Categories: Azure, Azure Cognitive Service, Azure Stream Analytics
Leave a Reply