Monday, August 7, 2017

BizTalk 2010 Failover Cluster Issue

Introduction

Failover clustering of BizTalk services is important for processes that require only a single BizTalk server to pick up a message from a source systems, particularly for transport mechanisms that do not have some kind of a locking mechanism such as FTP, MSMQ, POP3. In order for that to happen though, you need to configure a cluster for your BizTalk receive host. If you did not cluster the receive host, then there is a possibility that all the BizTalk servers could potentially pick up the same message at the same time, resulting in duplicate messages. When failing over from one node to the other the active node's host instances would stop and similarly the other node's host instances would start up.


Problem

A while back we were experiencing a problem on one of our BizTalk server (T03A) which required the operating system to be re-installed. This was all fine because we had our secondary test server (T03B) running, so it was safe to re-install T03A. The server was re-installed, all software was loaded, all the patches were successfully applied and the server was re-joined onto the cluster. Once BizTalk was installed, configured and re-joined to the BizTalk group, we started experiencing a strange behavior on T03A.

We started verifying the failover clustering between T03A and T03B servers to ensure that the failover works as expected. When checking T03B, everything looked good:


Though when we forced T03A to become the active cluster, I noticed that there was no (Active) labels even though T03A was the active server.

I then began to double check the failover cluster settings but all the settings appeared correct. I then began to search the internet for anything relating to this problem but I could not find anything that matched my problem. Lastly I started to check the configurations in WMI using PowerShell.

I opened a PowerShell with administrative rights and ran the following command on both T03A and T03B:

get-wmiobject MSBTS_HostInstance  -namespace "root\MicrosoftBizTalkServer" | Where-Object { $_.ClusterInstanceType -in (1,2,3) } | format-table HostName,HostType,ClusterInstanceType,ConfigurationState,RunningServer,ServiceState -AutoSize

T03A

T03B

Immediately, you can see the issue. The running server on node T03A is in lower case whereas the cluster node on T03B is all upper case.

Note:
The values in the tables can be found below.

ClusterInstanceType (https://msdn.microsoft.com/en-us/library/aa578062.aspx)

Value
ClusterInstanceType
0
UnClusteredInstance
1
ClusteredInstance
2
ClusteredVirtualInstance

ConfigurationState (https://msdn.microsoft.com/en-us/library/aa560498.aspx)

Value
ConfigurationState
1
Installed
2
Installation failed
3
Uninstallation failed
4
Update failed
5
Not installed


Value
ServiceState
1
Stopped
2
Start pending
3
Stop pending
4
Running
5
Continue pending
6
Pause pending
7
Paused
8
Unknown

Resolving the issue

To resolve the issue, you need to first evict the troublesome node from the cluster and then re-add it to the cluster.

Note:
You should never evict a cluster node from a cluster group because evicting it might actually cause your issue to become more serious and cause more harm than necessary. The following are the few exceptions to this:
  • Replacing a node with different hardware.
  • Reinstalling the operating system.
  • Permanently removing a node from a cluster.
  • Renaming a node of a cluster. 
Copied from this blog post: https://blogs.technet.microsoft.com/askcore/2010/03/03/when-should-i-evict-a-cluster-node/

Removing the node from the cluster

In our case we had to re-install the operating system as well as rename the node of T03A from lower case to upper case, so it was safe to evict T03A from the cluster.

1. Open the Failover Cluster Manager and connect to your BizTalk cluster.
2. Expand the BizTalk Cluster group.
3. Click on Nodes.
4. Right click on the troublesome node, "More Actions..." and then "Evict".


Adding the cluster node

One of the things that our infrastructure team did was to use the Failover Cluster Manager interface to add the T03A server back onto the cluster. When adding the server through the Failover Cluster Manger, the node would sometimes appear in upper case and sometimes in lower case. There is a way to force the the server to be added specifically in the way you want and that is to use a command line or PowerShell. See http://blog.workinghardinit.work/2012/04/26/failover-cluster-node-names-in-upper-lower-case-in-window-2012-with-cluster-exe-powershell-gui/.

Open a command prompt with administrative privileges and then type the following command:

cluster.exe /cluster:{CLUSTER GROUP NAME} /add /node:{SERVERNAME IN UPPERCASE}

Validate the cluster

Once the server was re-joined to the cluster group, you need to validate the cluster. See this article for more information on the validating the cluster: https://technet.microsoft.com/en-us/library/cc732035(v=ws.10).aspx

1. In the Failover Cluster Manager, connect to your BizTalk cluster.
2. Right click on the BizTalk cluster group name and then click on "Validate This Cluster...".


Once it passes, connect to the server with the cluster issue and open the BizTalk Administrative Console. You will now be able to see that the server has the "Active" labels on the server name.


Wednesday, July 26, 2017

PowerApps and BizTalk (Part 4 - PowerApps)

Introduction

In my previous post I demonstrated how to set up the security for your Azure API App using Azure AD. The Azure AD uses OAuth which forces any unauthenticated connection to the Azure API App to be authenticated using your Azure AD.

Note: 
The current configuration of your Azure API App is a single tenant only which means only users that are in your tenant or directory can access your Azure API App. In order to allow users from other tenants to make use of your API you would need to switch it to a multi tenant mode. This mode allows anyone from any tenant to make use of your Azure API App. Unless you have a specific reason to allow someone from outside your organization to start experimenting with the management of your BizTalk artifacts or ports, I would suggest you leave this in single tenant mode only.


4. PowerApps

PowerApps allows you to quickly create an application in the cloud which is accessible using mobile or a web portal. The design time of PowerApps allows you to design your application in a browser (https://web.powerapps.com/) or in PowerApps Studio which is downloadable via Microsoft Store.

Note:
Something important to note is that your Azure API App MUST USE the same organization/ school account as PowerApps. I.e if you sign in with example@company.com in Azure. Then when you sign into PowerApps you would also need to sign in or have signed up with example@company.com. This is related to single/ multi host tenants. Currently PowerApps is available only as a single tenant which means that your PowerApp cannot use an Azure API App from a different tenant.


Licensing

The first things you need to do in order to use PowerApps is that you need to sign in with your work/organization account or a school account. If you have an Office 365 or a Dynamics 365, you simply need to sign in/ sign up with PowerApps. See https://powerapps.microsoft.com/en-us/tutorials/pricing-billing-skus/ regarding the pricing.

Environments

Before we start working on creating our PowerApp, let's create a DEV environment to contain our BizTalk PowerApp.
1. On the top right of the toolbar, click on the settings button and then "Admin center".

This will open a new window containing the administrative page for you PowerApps.

2. Click on the "New environment" button.


3. Give a name for your DEV environment and a region then click on "Create environment".


The wizard will then start creating your DEV environment.
The next screen asks if you want to create a database.
At this point let's select "Skip" for now but to learn more you can browse to the Microsoft document site at https://powerapps.microsoft.com/en-us/tutorials/create-database/.


4. Click Skip.
Once you click on the "Skip" button, it will take a few minutes to get your dev environment up and running. When it's finally ready you will see it in the the Admin center under environments:


5. Close this window and in the previous window where you logged into PowerApps, refresh the browser.
6. Click on the drop down button next to environment and then click on your DEV environment:


Your development environment is now complete.

Custom Connectors

PowerApps allows custom connectors to utilize any public RESTful API as long as it has a well defined swagger definition. These endpoints can be either "http" or "https".

Let's add our BizTalk Management Service to the custom connector.

1. Browse to http://localhost/BizTalkManagementService/swagger/docs/v1 and save the swagger file as you'll need to make some changes to it.

2. Once you have saved it, open it using your favourite editor.
3. Change the host to your Azure Api Service website.
Host: biztalkappnamegoeshere.azurewebsites.net
Schemes: https
4. Save to your local machine.

If you are not signed into PowerApps, go ahead and sign in but ensure you are in your DEV environment.

5. Once you have signed in, on the menu on the left click on connections and then Manage custom connectors:



6. Click on create custom connector:


7. Click on the browse button and select your swagger definition you saved above (Step 4):


8. Select you swagger definition file and click on open.

You will then be able to see that the title of your PowerApp has now changed to "MicrosoftBizTalkManagementService" and scrolling down to the bottom of the screen, you will now be able to see that PowerApps is now referencing your Azure API App.

Note:
At this point you can rename the title of your connector as you see fit but once you save it, you cannot rename it.


9. Click on "Continue".
10. Select the drop down button and select OAuth 2.0:

11. Under OAuth 2, change the following values:
Field Value Description
Identity Provider Azure Active Directory We are using Azure Active Directory as our method of authentication
Client Id Client ID Azure AD Identity App - This is the guid of your identity app that you registered in Azure AD.
Client Secret Client Secret Azure AD Identity Secret - This is the secret of your identity app that you registered in Azure AD. See step 15 of my previous post.
Login URL https://login.windows.net Leave as is - this is the login location in which starts the authentication process
Tenant ID common Leave as is
Resource URL This is the guid of your app that you registered in Azure AD 
Scope Scope Leave as is

12. Click on Continue
Note:
The definitions contain all the operations based on the swagger definition file. Notice though that validations of the swagger definition has already been performed. These must be resolved before we can continue.



13. Selecting the first one, we can see the details of the operation which contains the error.
If we select the next error, which also contains an error we can see the cause of the errors:


As can be seen above, both operations have the exact same operation ID: "Applications_Get". According to the Swagger definition, the operation ID must be unique. (See https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#fixed-fields-5)
To resolve this, I looked at the summary particularly on the operation that specified "GET Specific Application" and changed it's Operation ID to Application_Get to make it more specific... pun intended. 😜

Once you make the change, click elsewhere or scroll further down and you will notice that the red error icon has now changed to a blue icon. The blue icons are good whereas red icons are bad...


Scroll down the operations and resolve all the errors. (All the errors are similar to the issue above and can be fixed by making the specific operations singular instead of plural.)

After resolving all the errors, scroll back to the top and then click on "Create connector".


Once your custom connector has been created, then you can click on the "Test" step. If you click on the test step before creating the custom connector, you will see the warning: "You must create the custom connector before testing."


14. After clicking on the "Create connector", click on the Test step.
15. Under connections, click on "New connection".


16. Clicking on the "New connection", you will get a popup window which will provide the actual connection to your Azure API App.


17. Click on Create.
You will now be prompted to sign using your Azure login.


Once, you have signed in you will taken back to the connections screen.

That's it, congrats and your first custom PowerApps connector.

Testing you custom connector

Now that you have your connector setup, let's test it out:
1. Click on "Manage custom connectors".
2. Click on your custom connector: "MicrosoftBizTalkManagementService" (or whatever you renamed it to).
3. Then click on the last step, "Test".
You should now be able to see the connection you previously made.


4. Scroll down, and click on the "Applications_Get" operation.
5. Click on "Test operation".


6. You should now be able to see a response from your Azure API App:


This means that you have successfully created you custom connection to your Azure API App secured using Azure AD.

PowerApps design time

Now that we have the connection setup, let's start designing the PowerApp with our new custom connection in our DEV environment.

1. On the main menu on the left, click on "Apps" and then click on "Create an App".


2. Choose a blank app.
Note: A good way of learning how to use PowerApps, is to take a look at other Apps and seeing how it's been designed and developed. There is also a wealth of tutorials that can be found on the internet providing step by step details on developing PowerApps.


This is the designer screen on the web page. If you want you can also download the desktop studio version. Both provide a very similar design time experience although I prefer the Studio version when adding complex functions as the processing is done locally on your machine rather than in the browser.


3. To add your custom connection, in the Screen pane, click on "Data" and then on "Add data source".


4. You should be able to see the connection to your custom connector that you previously made:


If  you cannot find it, click on the "New connection" button and then search for your custom connection and click on it:

5. Click on "Create":


This step creates your connection to your custom connector.
Once it's created, you now will be able to access all the operations of the BizTalk management API from you Azure API app.

Designing your PowerApps

As I mentioned before, there are many good tutorials on PowerApps out there. (See https://powerapps.microsoft.com/en-us/tutorials/getting-started/) Therefore I'll show a simple formula and then show how it can be used within PowerApps to show all your applications.

In your PowerApps you will be able to see the main screen as well as the menu options.
1. Click on "Action" and then "On Start":


This provides a very useful feature in PowerApps which allows you to initialize any collections or variables that your application requires.

2. Click on the formula bar and type the following all in one line:
ClearCollect(collectionApplications, 'Microsoft.BizTalk.ManagementService'.ApplicationsGet())

3. Click on the "Insert" menu and then click on Gallery and finally on Blank vertical.


4. Once the Gallery container appears, you should be able to see "Data" tab on the right. Click on the drop down button next to CustomGallerySample.


5. Then click on "collectionApplication".



6. Next, click on the drop down button in layout. Default is Blank:


7. Click on Title and subtitle.


8. Save your PowerApps (Ctrl + s) or click on File, Save and then save.

9. Close down your application by clicking on File, and then close.

10. Click on "Open", "PowerApps" and then finally your PowerApp.


11. You should then be able to see your BizTalk applications.


Alerts

In the event that a send port or receive location goes down, there are a few ways in which to send out alerts so that your BizTalk support team can investigate and take corrective actions.

Email alert

One method is to send an email to your BizTalk group with the link to the PowerApp. This will then require the user to click on the link to open the web PowerApp. To do this:

1. Log into your PowerApp and then click on Apps:


2. Click on the information icon or the ellipses and then details.


3. The next screen shows the Web link which is the link to your PowerApps.

4. An email can used to send alerts to your BizTalk operators in the event that a receive location or send port or even the host instance goes down. When coupled with some rules in SCOM or BizTalk360 an alert can be fired to your BizTalk support email group where the appropriate action can be taken.

Microsoft Flow with a PowerApps Push Notification (Premium Feature)

Another method is to use Microsoft Flow to send an alert out which makes the integration appear as prat of the operating system though the only caveat is that you must have the PowerApps studio installed on your desktop or on your iPhone/ Windows or Android phone. When an alert is triggered you would see the following alert:

Desktop:

On my iPhone:

Tapping the notification will automatically launch your BizTalk PowerApps application.

To set this up:
1. You will need to sign into Microsoft Flow at http://flow.microsoft.com/ using your organizational account.
2. Ensure that you are using your development environment that you had created earlier:


3. Click "Create from blank".


 4. Give your flow name an appropriate name that you can use to identify your flow later and then click on the Request/ Response trigger card:


5. Click on "Request/ Response - Request".
6. Click on "Use sample payload to generate schema", type the following and then click "Done":
{"applicationName": "BizTalkApplication"}
7. After you click "Done", a schema based on the payload data will be generated which we will use later.
8. Click on "New step" and then "Add an action":


9. Type "PowerApp" and then click on the "PowerApps Notification - SendPushNotification" action card:


10. You will now need to configure the connection details for the PowerApp. In the "Connection Name" type: "BizTalkManagement PowerApp".

11. "Target application" can be either the full URL to your PowerApp or the ID of your PowerApp. I have already shown how to get the URL above. The ID is just under the Web link:


12. Once you copy and paste the App ID into the "Target Application", click "Create":


13. The recipients should be the list of BizTalk operators that would be maintaining the BizTalk servers.

The "Message" is the message that will appear in the Push Notification, so type in:
"Alert encountered on BizTalk Application: " and then click on the "applicationName" variable 
content from the JSON schema.


Open App simply means that the App will be opened when the notification is tapped.

Parameters: I won't go into too much details here but here you can specify additional information to be passed to the PowerApp such as the BizTalk Application and artifacts that is down. For example a SendPort1 in the BizTalkApplication1 Application could become disabled which generates the push notification. When the push notification is tapped normally the PowerApp would only open up but you would still need to browse to the Application that has generated the alert. Using the parameters you can write code to detect for these parameters and then automatically navigate the user straight to the BizTalk Application thus saving the user precious minutes searching for the BizTalk Application.

14. Click on "Save flow".

Once saved you can test this out using Postman.

Testing

1. Change the verb to "POST". 
2. Copy the URL from the Request in the flow and paste it into Postman.
3. Add a header "Content-Type" with the value "application/json".


3. Next click on Body and enter the following:
{"applicationName": "BizTalkApplication"}

4. Click "Send". 

This will send the request to Microsoft Flow which will trigger the Push Notification alert to the recipient specified. You could also make the recipients as part of the JSON payload in step 6 above, which will allow the Flow to dynamically send to several recipients.

At FortisAlberta we are using BizTalk360 to monitor our BizTalk Servers so to generate the alert, we can use a custom notification channel to trigger an HTTP call to our Microsoft Flow in order to generate the PowerApps Push Notification. An alternative is to use SCOM with PowerShell to invoke the request to our Microsoft Flow.

Conclusion

In this post, I showed how to create a development environment for developing PowerApps as well as how to create a custom connector which links to your Azure API service secured by your Azure Active Directory. I then showed how this is configured and then tested to ensure that the call from PowerApps to your Azure API service is functioning correctly. Then I demonstrated how to design your PowerApps application to invoke the call to you Azure API App and finally the steps to create a control to display your BizTalk applications. 

Performing all of the above steps takes no more than a few minutes and provides for another alternative to the BizTalk Administrative Console all from your own Windows/ iPhone/ Android phone with the addition of alerts to notify you when an artifact is down.