In software, the only thing that is constant is change. And software architecture is no different, it has to evolve. Simply put, system components should be organized in such a way that they support constant change without any downtime, Even if all the components are changed or completely replaced, the show must go on.
Software architecture has to be technology agnostic, Resilient, and designed for the incremental change.
If we think about the Theseus paradox, Evolutionary architectures care less about the sameness of the ship(s) and focus more whether the ship would keep on sailing. In short, architecture has to be technology agnostic, resilient and designed for the incremental change.
We can call any system “evolutionary” if it is designed for incremental change and can evolve with the changing nature of the business. Microservices style architecture is considered evolutionary because each service is decoupled from all other services in a system and communication between services is completely technology agnostic, therefore replacing one microservice with another is rather easy.
Microservices architecture focuses more on the technology neutral communication between the components, loose coupling, versioning, automated deployment etc. but it doesn’t say anything about how each service or a complete system should store data internally.
Event sourcing (ES) pattern can serve as the missing piece in the puzzle, by defining data storage strategy.
Butchering the data is a criminal offense
In many businesses, modifying or changing the data is a crime, for example, Banking, forensic, medical, law etc. Remember Accountants don’t use erasers. Whether we like it or not, for certain businesses like Telecom and ISP retaining data for a period of time is compulsory.
Can you imagine CCTV systems without recording? what will happen if something happens and you have no record of it?
Most serial killers kill for enjoyment or financial gains.
In the early 1990s, Dr. Mike Aamodt started a research about serial killers behaviors. It took him and his students decades to study and analyze public records before they could answer why and how serial killer kills? this wouldn’t have happened without historical records. In order to answer future questions, we must persist everything that is happening today.
Don’t create serial killers of the data.
We all know that the currency of future is data. We don’t know what tools and methods we will have at our disposal in a decade, we certainly can’t predict how important that data could be for our businesses or for us personally, Still, we design and develop applications that are constantly murdering valuable data.
We as a species accumulate memories from our experiences, We simply can’t think like a CRUD system. There is a reason we love stories and pretty much everyone hates spoilers, Our brains work like an ES system.
Event sourcing is a way of persisting domain events (the changes that have occurred over time) in your application, that series of events determine the current state of the application. ES and CQRS go hand in hand, We simply can’t talk about ES without mentioning the CQRS (Command and query responsibility segregation). I generally explain CQRS with a notion of one-way and two-way streets or separate paths for pedestrians, cycling, and other traffic.
let’s see how Greg Young himself explains it.
CQRS is simply the creation of two objects where there was previously only one. The separation occurs based upon whether the methods are a command or a query (the same definition that is used by Meyer in Command and Query Separation: a command is any method that mutates state and a query is any method that returns a value).
—Greg Young, CQRS, Task Based UIs, Event Sourcing agh!
The advantages of using CQRS/ES
- Complete log of changes
- Time traveling (replaying events is like storytelling)
- Easy anomaly and fraud detection (by log comparison)
- Death of cannot reproduce or Easy debugging (production data can be debugged in dev environment)
- Read (with CQRS) and Write performance
- Scalability (with CQRS)
Since Event sourcing enables you to audit state changes in your system. Auditors will love you for this. Historical records can be used to perform analysis, and identify interesting patterns. We must consider ES when accountability is critical (e.g Banking, medicine, law).
If your system by nature is Event-driven then using ES is a natural choice (e.g game score tracking, online gaming, stock monitoring, real-estate, vehicle tracking, social media like twitter, facebook, LinkedIn etc).
You should consider this pattern If you need to maintain versions or historical data for example document handling, Wikis etc.
Similarly, after each change, we can replay all the events and inspect the logs to see if expected change is there? and does it work?
Most people think embracing ES is complex, however, I believe in the opposite. I think ES systems are easy to scale and change. All the mature industries like Banking, Finance, Law, and Medicine already use these methods, even before computers were invented.
We simply can’t destroy the data, because we can’t predict what tools will be available in a year from now, and how useful this data would be in future.
In a forthcoming blog, we will show you a production-quality implementation of a system that has the potential to evolve.
If you happen to be in Stockholm and have an irresistible urge to discuss evolutionary architectures such as CQRS/ES lets have a “Fika”.
At byBrick Development the culture is focused around the sharing of knowledge.
To that effect we run a program called byKnowledge and as part of byKnowledge we have a special session type called Pecha Kucha.
Pecha Kucha Format
The format is very simple and invites 4-5 different presentations from individual members of byBrick Development.
- Session length is 7 mins in duration
- Topic can be anything, including non-technical
The idea is to showcase and share knowledge in a very succint manner. It is very effective and also allow us to get to know out colleagues a little better.
The byKnowledge, and by extension Pecha Kucha events run monthly for us and we continue to improve on the format and style.
The topics from last night
We had a great range of topics last night, the focus here was more technical in nature, but previously we have even had “Stress Management” on the list.
- Agile Scrum – Mr. Osman
- Serverless Functions – Mr. Tuohisaari
- Crypto & Blockchain – Mr. Hassan
- Evolutionary Architecture – Event Sourcing – Mr. Sajid
- Code Measures and VSTS Data – Mr. Madsen
Benefits of sharing knowledge
We work in an industry where it is nigh on impossible to be aware of every single aspect.
In order to gain an incremental curve of learning and knowledge, we work through our own experiences and share it with our colleagues.
Aside from attending conferences, meetups and training we gain exponentially by collectively sharing the information we gain.
As the opening stated, it is heavily embedded in our culture and it’s a testament to the people we work with that we get the opportunity to learn from their real life experiences.
CFO asks his CEO: What happens if we invest in developing our people and then they leave the company?
CEO answers: What happens if we don’t, and they stay?
I know!! it’s a meme that’s done the rounds for years now – but honestly, it’s a paradigm which is incredibly accurate in its message.
Creating a culture where focus is on giving our people the ability to change and adapt. It’s not something which is all that common but it’s a strong focus for us.
Luckily our consultants have embraced this and we continued to adopt and embrace new ideas – it is the core of what byBrick is all about.
Second phase of BizTalk Management and MessageBox DB sync issue is Un-Deploying/Deleting application. Fortunately issue was with only 2 applications.
When tried to Delete (either from Cosnole Or BTSTask -Remove App command ) an application in order to deploy a new version,we got an error popping out every-time saying
“The service action could not be performed because the service is not registered”
The application was visible in the BizTalk Admin Console.
Funny part was; application was running without any issue and all messages subscribed by artifacts of this application were being processed successfully.
This helped me in buying some time to dig out the issue.
From previous experience it was evident that something was missing in the BizTalk Databases and most likely in the BizTalkMsgBoxDb and analyzing further with help of SQL-Profiler and BizTalk Trace we were able to find that Port information was not present in Service table and Application name as not registered in Module table.
Take backup of BizTalk Databases (we can rely on SQL jobs for BizTalk DBs 🙂 )
Stop all hosts instances.
Check application artifacts in following tables. bts_application, bts_receiveport, and bts_sendport and bts_assembly tables in the BizTalkMgmtDb. (If you have other artifact in your application, then you need to check in respective table e.g. Send Port Group, Orchestration etc.
Insert Application Name in Module tables which will (auto)generated Module Id. Use this Id ‘nModuleID’ in service tablle
Insert into [BizTalkMsgBoxDb].[dbo].[Modules] ([nvcName],[dtTimeStamp]) values ('Application.MyApplication', GETDATE())
Find Service Instance IDs (uidGUID) in bts_SendPorts, bts_ReceievPorts i.e. (all possible artifacts tables).
SELECT uidGUID FROM [BizTalkMgmtDb].[dbo].[bts_receiveport] WITH (NOLOCK) WHERE nvcName =<'your artefact name'> SELECT uidGUID FROM [BizTalkMgmtDb].[dbo].[bts_sendport] WITH (NOLOCK) WHERE nvcName =<'your artefact name'>
Then insert Service Instance ID (uidGUID) in Services table with Module ID for all artifacts One by One
I had 2 Receive Ports and 2 Send Ports which led me to make 4 entries in Services table.
Get ModuleID from Modules table for respective application and use in below query
Insert INTO[BizTalkMsgBoxDb].[dbo].[Services] ([uidServiceID],uidServiceClassID,[nModuleID],[fAttributes]) VALUES ('<ServiceIntanceID from artifact table>',Null,<ModuleID>,0)
Un-Deploy Or Re-Deploy Happily.
While using BizTalk Clustered Environment with Availability Groups (Not Failover Cluster), an issue was found while deploying BizTalk Application.
BizTalk Management and MessageBox Databases goes out sync (due to some specific or unknown reason), we get error while deploying an application. (it could be 1 or more applications; in my case it was only one)
Deploying another version of same application in BizTalk Environment creates issue.
It is noticed that there are activation subscriptions left even after Un-deploying the BizTalk application. Its impossible to deploy the BizTalk application again with Orphan Subscriptions.
Executing AddApp command, it gives error
Error: Application registration failed because the application already exists
Executing RemoveApp command for same application , it gives error
Error: Application ” M ” not found in configuration database.
This proves there is contradiction in BizTalk Database instances.
Event Log produces generic log.
Unable to communicate with MessageBox BizTalkMsgBoxDb on SQL Instance ABCD11111\MessageBoxInst. Error Code: 0x8004d00e. Possible reasons include:
1) The MessageBox is unavailable.
2) The network link from this machine to the MessageBox is down.
3) The DTC Configuration on either this local machine or the machine hosting this MessageBox is incorrect
- Possibly, Maximum Degree Of Parallelism for MessageBox Database (MDOP) was not set to 1. It was grater than 1 during this period of time which have caused database issues when databases are Synched in AG.
- Orphan Subscriptions of application.
- Orphans of application.
Steps for Tracing the issue
- Executed command BTSTask ListApps
- Executed command BTSTask ListApp /ApplicationName:”M………”
- Check Test DTC settings.
No trace of application found.
Then the last option left is to dig in BizTalk Databases.
- Run SQL profiler on MessageBox and Management databases while executing BTSTask commands and collect the trace log.
- Run BizTalk Trace using “PSSDiagForBizTalk” while executing BTSTask commands
Try to search for application name in both trace logs and identify stored procedure or process creating issue.
This issue appears in MessageBox database due Orphans of Application. Execution of one stored proc i.e. bts_AdminAddModule got interrupted during deployment and application was not deleted from Modules and Services tables. Modules and Services tables in MessageBox are mainly responsible for subscriptions.
Note:- Do Not forget to stop Hosts while performing below activity.
- Delete Orphan Subscriptions using subscription IDs. (Using SQL Query or a tool you prefer)
- Run below SQL query and get nModuleID of application creating issue. SELECT * FROM [BizTalkMsgBoxDb].[dbo].[Modules] WITH (NOLOCK)
- Check for Instance in Application and Instances tables for nModuleID. Query result must be empty.
- Run below queries BEGIN TRANSACTIONDELETE FROM [BizTalkMsgBoxDb].[dbo].[Services] WHERE nModuleID = 111DELETE FROM [BizTalkMsgBoxDb].[dbo].[Modules] WHERE nModuleID = 111ROLLBACK TRANSACTION
Create SQL database on Azure using PowerShell and access from on Premises Microsoft SQL Server Management Studio
SQL Azure is cloud database-as-a-service provided by Microsoft cloud. The best part is that the data is hosted and managed by Microsoft data centers. We can build our applications on premises and move our data to cloud.
Azure SQL model provides:
- Elastic database pools – Allows you to provisioned 1000’s of database table as you need. Grow and shrink based on your requirement.
- Azure hosted database
- Pay for only what you use (no of plans available)
- Also enable Geo-replication for disaster recovery
- Reduces hardware cost and maintenance
In this example, we would be creating an SQL database on Azure Cloud and try to access SQL Server from on-premises SQL server management studio. We would be implementing this by using the PowerShell command and later configuring firewall rules that allow access from external parties.
- Azure subscription
- Azure PowerShell modules version 4.0 and later (Azure PowerShell Web API)
Once we are ready with the PowerShell modules. Open Windows PowerShell ISE (x86) as an administrator.
- Login to your Azure portal using the below command.
Apply necessary login details and click OK. After successful login,
- Lets create a Resource Group.
$ location = "west tour" $ resourceGroup = "mysqlserverRG" New-AzureRmResourceGroup -ResourceGroupName $ resourceGroup -Location $ location
- Let’s create a Logical SQL Server. We would be using “New-AzureRmSqlServer” command. A Logical SQL server contains a group of databases managed as a group.
$ servername = "myblogazuresqlserver" $ username = "mypersonallogin" $ password = "xxxxxxx" New-AzureRmSqlServer -ResourceGroupName $resourcegroupname -ServerName $ serverName -SllAdministratorCredentials $ ( New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $ username , $ ( ConvertTo-SecureString -String $ password -AsPlainText -Force ))
Once the Logical Server is in place.
- Next step is to create a database within that server.
$ databasename = "myBlogDatabase" New-AzureRmSqlDatabase -ResourceGroupName $ resource_groupname -ServerName $ serverName -DatabaseName $ database name
Let’s navigate to the Azure portal and check the database we created “myBlogDatabase”. It uses a standard pricing tier.
- DTU: 10 (S0) “Database Transaction Units”
- Storage: 250 GB
Now when you try to log in to Cloud SQL Server from your own premises – SQL server management studio. You will receive an error message as below.
Navigate to the Azure portal and configure the firewall rules to allow access to your on-premises server.
Navigate to your database created in cloud -> Click Firewall setting -> specify the Rule name and Start and End IP. Once the rules are in place, you can now successfully login to Azure SQL Server from on-premises SQL Management Studio.
Resiliency in Distributed Systems
- General network connectivity intermittency or temporary failure, i.e. Outages.
- Spikes in load.
- Basic network dependencies like Database Servers.
- Systems integration and distributed processes.
- SOA based Applications
- Zero dependency, it is only Polly.NET Packages.
- Fluent API, supports both synchronous and asynchronous operations.
- Targets both .NET Framework and .NET Core runtimes.
- Identify what problem to handle, this will likely translate to an Exception in .NET.
- Policy or Policies to use.
- Execute the Policy or Policy group.
// Simple Retry Policy, will make three attempts before giving up. var retryPolicy = Policy .Handle() .WaitAndRetryAsync(3); HttpResponseMessage response = await retryPolicy.ExecuteAsync(() => httpClient.GetAsync(uriToCheck));
// Retry with a 'Wait' period which is given as TimeSpan in thte second argument to WaitAndRetryAsync var retryPolicy = Policy .Handle() .WaitAndRetryAsync(3, (r) => TimeSpan.FromSeconds(r * 1.5f)); HttpResponseMessage response = await retryPolicy.ExecuteAsync(() => httpClient.GetAsync(uriToCheck));
Now, in this version, we can see an extra lambda function returning a TimeSpan which represents the time we want to wait before each subsequent attempt, if we dissect that line we can find the following:
- The argument r passed to the lambda represents the attempt number, in this case it will have three possible values: 1,2 and 3.
- We multiply the attempt number by a float constant greater than 1, this gives the effect of an increasing delay on each attempt.
Applying a Strategy
// Exponential back-off behavior var retryPolicy = Policy .Handle() .WaitAndRetryAsync(3, (r) => TimeSpan.FromSeconds(r * 1.5f)); // Degrade gracefully by returning bad gateway var fallbackPolicy = Policy .Handle() .FallbackAsync((t) => Task.FromResult(HttpStatusCode.BadGateway)); /** * Combine them, first argument is outer-most, last is inner-most. * Policies are evaluated from inner-most to outer-most, in this case: * retry then fallback */ var policyStrategy = Policy.WrapAsync(fallbackPolicy, retryPolicy); HttpStatusCode resultCode = await policyStrategy.ExecuteAsync(async () => (await httpClient.GetAsync(uriToCheck)).StatusCode);
We are dealing with status codes here for simplicity of the example, but a better approach for the fallback action would be to follow a Null Object Design Pattern and produce an empty list of Users, this really depends on the requirements and structure of our Application. The patterns we have introduced in this article are only a few I have picked to introduce you to how the library works and how such patterns can be applied. Polly exposes even more Policies to enforce a broad set of well-known software design patterns for resiliency such as:
- Circuit Breaker.
- Advanced Circuit Breaker.
- Bulkhead Isolation.
- Cache (which gives us the cache-aside / read-through pattern)
For the most part, our lives aren’t that particular complicated. Our decisions don’t always have the impact of a meteorite in our lives or others. But some decisions do tend to follow us, nay, haunt us for a longer period of time than we originally anticipated – namely the choice we made when settling on using a specific tool.
Some years back my wife asked me to build her a veggie garden out back, with a picket fence, gate, paved path, a simple shed and sandstone borders. It was a simple setup until I got to the stage where I needed to build the gate.
Should in passing here mention that my handyman skills are pretty much trial and error or whatever tips and tricks I can locate on the web (read: YouTube, Vimeo et al – I need pictures too for this sort of work).
Rather than simply buying a finished gate, I had to go manly and build a custom one. After all, my skills had recently grown to include the level of a master carpenter, no? The picket fence, veggie garden et al was nearly complete.
After struggling for 8hrs more than was necessary, I finally had the gate finished and needed to mount it. Of course, it didn’t fit so I had to dismember the gate again. This went on for a little while till I finally got it right.
The lesson I took away from this was, at the time (of course), that carpentry wasn’t my main forte so I could be excused for taking a little longer, producing a gate that wasn’t fitted professionally and a small gap existed between the fence and sandstone bricks for the borders.
Looking back, the first mistake was obviously the choice of tool to use – namely myself.
Facing choices in business
Moving along, to an industry where such choices pop up on a near daily basis, the ramifications of making the incorrect choice obviously has a steep impact on profitability and business continuity.
The IT&T industry is ripe with these pitfalls, both from the choice of service provider, the hiring process and down to which framework to adopt for a given project. If you ask a practitioner for advice on what product, client tools or enterprise framework to use, self-interest will always kick in and the answer is what preserves the current status of the practitioner or which will enhance it.
The same would be said for having so-called independent consultants provide you with a recommendation. Said consultancy would naturally focus on an answer which would provide them with the highest percentage chance of continuing the engagement – would that make the answers and recommendations wrong or inaccurate? Of course not, not by those virtues alone.
This is of course what makes it even harder. Why can’t you trust an independent recommendation?
Simply because there are a thousand ways to skin a cat. What is the correct choice today, will likely change in the near future as the business change.
Buying off the rack
Information Technology adoption decisions tend to be made, firstly, by the financial impacts of adoption. For a business, which doesn’t deal with IT but is largely dependent on it, the bottom line will generally be cost and the equation simple.
x => y
Where x = budget and y = cost.
Of course, the relevant departments will have done their due diligence with regards to product choices or service provider. They’d have had a string of sales and technical presales consultants strut their stuff, showing off their product or services in the best light possible.
People far smarter than I have been telling us about this for years already. So why is it, that IT projects have one of the largest failure percentages in the world? Why is it, that budgets are only ever guides, not true costs of adoption? Surely we should have learned enough by now to know how to do this right.
It’s not always simple to make the right choice. Even for subject matter experts who make a living working in a space littered with off the shelf solutions, talented developers, and magicians.
When working through the requirements, we all know that it’s important to engage in depth with stakeholders and business users. Buy-in from the business is mandatory for adoption or project success. Without it, you simply won’t ever be able to complete the project with a positive outcome.
Words of wisdom that has been known for decades to be immutable facts.
Taking on a custom solution
However, for project success, it’s not just about delivering on time and under budget and there are many more factors that have to be weighed before the stamp of approval is given.
This is especially true when the choice comes in to create a custom solution – a solution which suits your business as a deerskin glove…smooth and comfortable.
One of the aspects, which helps define the cost of adoption for software projects, is calculating technical debt.
Ted Theodoropoulos wrote an excellent article, back in 2010, where he goes into the primary points needed in order to identify and calculate technical debt.
I can highly recommend reading his 4 part series on Technical Debt (http://blog.acrowire.com/technical-debt/technical-debt-part-1-definition).
Accurately predicting the future
Obviously, it’s not feasible to accurately predict the future. I’m sure the world would be a very different place right now if that was not the case.
We spend a lot of effort and money on analyzing the operational paths, which could potentially, with some degree of probability come true…maybe… Business Intelligence analysis can show you predictive trend analysis outcomes, Technical Subject Matter Experts could tell you what direction the market is moving in and your staff could give you their opinion based on past experiences.
When the chips are down, the fact is…we just don’t know.
So since we can’t predict the future and haven’t quite mastered time travel yet, what can be done about it?
The most important aspect of any decision is that it cannot be definite. We know the future can change week by week, so why are we so set on ensuring that our decisions are definite?
How often have we seen the following play out after 2 years of product/project implementation?
Director: “Why did you choose x Product/Platform as the solution for our ERP. It has now cost us $X million so far, with no end in sight?”
IT Manager: “it was within our originally estimated budget and covered about 80% of our immediate business requirements 2 years ago, but that was before we expanded our service offering and went international. We didn’t know these changes would happen when the purchasing decision was made.”
The predictive nature of business risk calculation always looks to past trends, and if found incorrect the crystal ball is hidden away and we’re stuck with the same moniker that plagues our industry – the 20-20 hindsight.
One size fits all
Off the shelf products (or shrink wrap products) rarely fits all industries, every business within and we generally pride ourselves on “doing it different than all the rest”. So why should a product ever be expected to fit your company processes perfectly? Just because we told the sales rep what we wanted and was told that it could cover all of them?
No, that’s just not a realistic answer or solution. When implementing an existing product, the business has to expect some level of change being needed internally. It might vary all dependent on the size of the implementation – the more areas it impacts on, the higher likelihood there is for it needs to change.
Choosing solutions that end up with more external work being needed, than what it covers, is obviously not a good investment.
An agile business needs agile decisions
The most important aspect of business today is that it needs to move with the needs of its customers and the market it operates in.
This expectation of agility needs to be applied to all facets of the business. Especially when it comes to IT investments and adoption. Being agile in the ways that your information is consumed, orders being processed and tasks completed comes down to how the systems are designed, from the bottom up.
Using cloud services for key integration points could be vital for your IT investments ability to support an expanding business.
Being able to facilitate information sharing across international locations could be essential for business growth, IP discovery, and collaboration. The architecture around systems needs to be built with flexibility in mind – the same goes for the platforms and tools used to build individual components.
That’s why it’s important to use the right tool for the right job…even down to the level where lines of code are written and what framework is being used.
At byBrick Development we are skilled in ascertaining best fit, by applying a methodology to the decision which adheres to the architectural principles you have established;
If your company doesn’t have principles in place then we can certainly also assist in developing those.
Why publish a blog?
This is probably a question a lot of non-techie companies would ask and this post is here to explain why we, at byBrick Development, decided to publish a blog.
First of all, we have a group of extremely creative and highly competent technical practitioners and what better way for us to share some of this knowledge than to write a blog about it.
Our core competencies are focused on Office 365, Integration and bespoke .Net solutions where many of our consultants have in excess of 10, 15 or even 20 years experience in the industry.
We are a highly diverse group of consultants, with people from all over the world, which
is yet another particularity about byBrick Development that sets us apart from many other consultancies.
Another particularity is that many of our consultants are actively engaged in knowledge sharing and believe strongly in it. So much so that it has been on their wishlist for quite some time that we set up a blog.
So what can be expected from this blog?
Seeing as there are a huge amount of experience amongst our consultants, it’s really very hard to specify just a single (or even a handful) of topics that will appear here. We promise that we will ensure that we have a great deal of spread across all of our technical and business knowledge.
But who are we?
The industry’s best IT Consultants…
We have chosen to focus on a small number of technologies and approaches in order to offer the best IT consultants in the areas where we operate. Our consultants are characterised by a unique ability to combine deep technical expertise with an understanding of various business needs and challenges.
IT should be as simple and cost-effective as possible. We work with proven platforms and standards so avoid locking our customers in systems and solutions that only we, or a few other providers can maintain and develop further. Our development methods provides full transparency with our customers, from costing to project management and status.
Our customers choose us because we are the best at what we do!