Better software through software architecture and devops

@jamessnape

Tag Archives: #domain-driven-design

  • January was a long month so I’ve got quite a list for you. I may consider doing these more often if readers think there are too many items for a single list.

    Governance

    Self-Service Business Intelligence Governance - Essential reading/watching for anyone planning to deliver self-service business intelligence.

    Five Stages of Data Grief - we’ve all been through this, “If you don’t think you have a quality problem with your data you haven’t looked at it yet”.

    Functional Programming

    Maybe that shouldn’t be settable - Bringing some of the F# Option type goodness into a C# world.

    Software Process

    Five Tips to Get Your Organisation Releasing Software Frequently - my team score well on these but culturally I can see some being quite difficult to implement, particularly around the devops style organisation of teams.

    Pairing vs. Code Review: Comparing Developer Cultures - pros and cons for each style of quality culture. Which, if any, is best?

    Is Agile BI Really a Better Mousetrap? - A great article on the benefits of agile BI. This really appeals due to its use of development process business intelligence - measure and optimise just like we preach to our customers.

    Using Vertical Slicing and Estimation to make Business Decisions at Adobe - A good look at the release planning process at Adobe with some nice techniques discussed.

    Personal Development

    Of Orcs and Software Craftsmanship - Best quote of the month if you are a parent: “These are the types of error messages that make debugging a software like debugging a 2 month old baby.”

    Yak Shaving Defined - Sometimes if feels like this all day long in software.

    Organisational Behaviour

    Performance Reviews Are Not Useful; Feedback Is - Personally I think performance reviews are something that human resources departments mandate; feedback is something that leaders give.

    If Managers Don’t Give Performance Reviews, What Happens? - Well, as it turns out, a lot of good things start to happen.

    Top 10 ways to ensure your best people will quit - some common mistakes; how many have you come across?

    Testing and Test Driven Development

    These next three links are related and if you read the first you should also read the second and third.

    The Failures of “Intro to TDD” - Justin Searls rips into the current way of teaching test driven development.

    The Domain Discontinuity - Bob Martin responds comprehensively but ends with why the issue is not about test driven development but wider issues such as architecture and domain design.

    Commentary on ‘Roman Numerals Kata with Commentary’ - Ultimately you must understand your domain before trying to do test driven development.

    Databases

    Default Configuration of SQL Server - Like most software, out of the box SQL is configured for the most general case and may need extra tuning for specific workloads. Thomas gives a simple set of extra configuration changes and reasons why. Also love the quote “If you are working in a bank, they may not apply to you.”

    Data Visualization

    Announcing Power BI for Office 365 - In case you missed it, all the fancy new BI capabilities in the Microsoft cloud are publicly available now. Shame we are stuck using corporate infrastructure.

    Famous Movie Quotes as Charts - A fun look at communication in chart form.

    Ten Tips and Tricks for New Tableau users - A rather nausea inducing format but useful tips for making great Tableau dashboards.

    Power Tools for Tableau - Desperate for some sort of an API with Tableau? This may be the answer.

    Statistics and Data Analysis

    Revolution Analytics - Want to run ‘R’ statistics against your Hadoop data? This seems to be the way to do it…

    Learn R interactively with the swirl package - It looks like R is going to be an important tool for us so anything that makes it easier to learn is a bonus.

    Learn Data Science Online with DataCamp - Similarly, learning data science online and interactively.

    Analysis of Health Inspection Data using F# - Another great example of using F# (and D3) to analyse data quickly and easily.

    Big Data

    Big Data: The organizational challenge - Some interesting stats comparing companies with the best analytic capabilities vs. those that don’t.

    Update on Stinger: the view from a Microsoft Committer - Stinger is the Hortonworks initiative for faster SQL queries against Hadoop. This article describes some of the recent performance gains.

    How To Install Hadoop on Windows with HDP 2.0 - Get Hadoop running on Windows with a minimum of fuss. However, our local Hadoop expert recommends you only do this at home; in the enterprise just setup a proper development cluster.

    How To Use Microsoft Excel to Visualize Hadoop Data - Tutorial for visualizing Hadoop data in Excel/PowerView, this one is for stock quotes.

    How to Visualize Website Clickstream Data - Another Hadoop tutorial this time on web click-stream data.

    50+ Open Source Tools for Big Data - I think one of the problems with open source is it littered with cute names that do little to describe software function so here is a useful list to help you distinguish the likes of Orient, Flock, Storm and others.

    Building your own web analytics system using Big Data tools - Should you build these things yourself? What are the choices? Are there any risks?

    Master Data the noun in Big Data sentences - I often talk about master data and spend more time worrying about dimension design than facts. It is useful to see how this applies to big data too.

    You don’t have big data… - With all this talk of big data it is worth remembering that most use cases do not quality at big. Most likely you have ‘hot data’.

    This entry was posted in reference  and tagged #apache-hadoop #domain-driven-design #microsoft-excel #tableau #test-driven-development #testing  on .
    Discuss this on Twitter or LinkedIn
  • Whilst researching the previous article I came across this link on Acronyms and Ubiquitous Language. It is well worth reading as everything discussed also applies to dimensional models. There is one quote that I want to reprint from the .NET Framework General Naming Conventions:

    Do not use any acronyms that are not widely accepted, and then only when necessary.

    Your business users should be able to point Excel (or whatever tool you are using) at a dimensional model and intuitively know what the measures, dimensions and attributes are because they describe the business your users work in. Since acronyms obfuscate meaning they don’t belong in dimensional models.

    The only time I generally relax this rule is when both the following are true:

    • All business users know the meaning of the acronym
    • The expanded version is so long that it becomes unwieldy
    This entry was posted in business-intelligence  and tagged #dimensional-model #domain-driven-design #naming-conventions #ubiquitous-language  on .
    Discuss this on Twitter or LinkedIn
  • Managing complexity graphic showing DDD patterns to apply from Domain Driven Design source https://commons.wikimedia.org/wiki/File:Maintaining_Model_Integrity.png

    My commute is around two and a half hours each way so I read a lot on the train. One of the subjects I’ve recently become interested in Domain Driven Design or DDD. I’ve found it isn’t really a new topic for me but more like someone has documented many of the techniques I’ve always used.

    DDD discusses data warehouses primarily as an output or reporting function within a larger application. The book Implementing Domain-Driven Design by Vaughn Vernon mentions reporting repeatedly as a by-product of DDD (particularly when used with Event Sourcing) but not directly as a possible use case.

    I would agree that not all concepts can be reused in data warehouse solutions since the only interface available is often one that transfers a set of mutations (property value changes) without the accompanying reasons (it is a key idea in DDD that you need to design your model not by changes in attributes but by operations performed on entities). For example an order count has decreased and the reason is missing – was the order returned, cancelled, an error etc. So where can it be applied? Are any of the concepts useful when designing data warehouses?

    Ubiquitous Language

    Lets start with one of the core concepts – Ubiquitous Language is a rigorous shared language used between developers and users. It is used to make sure that conversations are accurate and productive. It should evolve as the team’s understanding of a domain changes. The ubiquitous language is what forms the domain model at the heart of a software solution.

    I find this description very similar to an equivalent concept in data warehousing – the Dimensional Model. This model, and its associated dimension bus matrix, is based on real business processes and terminology. The dimensional model is the public face of the data warehouse. It needs to be precise, reflect the terms used by business users and form a common vocabulary between users and the development team. For example when browsing an Analysis Services cube in Excel, the dimensions and facts defined in the dimensional model are directly visible to end users – if they don’t automatically understand what is on-screen then the model doesn’t describe the business.

    Entities and Value Objects

    There are two types of object in domain driven design – entities and value objects. Value objects are immutable and identified via their attributes. For example $100 USD in one object is the interchangeable with $100 USD in another. Entities on the other had cannot be identified purely by their attributes – there must be some sort of unique identifier (in data warehouse terms this is a business key) to differentiate similar entities. For example, one John Smith may not be the same as another and need a Customer-Id to differentiate the two.

    With respect to dimensional models, value objects should not be implemented as top-level dimensions but instead be added at sets of attributes to the entities which own them. For example ‘Product Colour’ is a value type (colour) and should belong in the ‘Product’ dimension. This seems obvious when written this way but happens a lot.

    Entities and Aggregate Roots

    DDD groups sets of closely related entities under the control of a single ‘Aggregate root’. Some entities make no sense unless the parent entity is also within context; order lines and parent orders are the typical example.

    So value objects shouldn’t be dimensions and I don’t think ordinary entities should be either. True dimensions are the aggregate roots and the one thing that seals it for me is that an aggregate root (according to DDD) defines a transactional boundary – you should not update multiple aggregate roots within a single transaction; instead sagas keep your data warehouse in sync (eventually).

    Finally

    Domain driven design and business intelligence share a number of common concepts yet the two philosophies are rarely seen as related. I think there is a lot to be gained by applying software concepts from different viewpoints which may not ordinarily be considered.

    This entry was posted in business-intelligence  and tagged #data-warehouse #domain-driven-design  on .
    Discuss this on Twitter or LinkedIn