In the first article, we created Azure Cosmos SQL API with some sample data. Later, utilized the Azure Cosmos SDK within a .NET Console application to carry out fundamental CRUD operations.

The subsequent article illustrated the process of migrating data from Azure Cosmos SQL to an Atlas Cluster using some tools.

In this article, we will look at some of the key aspects of application migration when moving from CosmosDB SQL to Atlas.

Aspects of Application Migration

When migrating from CosmosDB SQL to MongoDB, there are several application aspects that should be considered to ensure a successful transition. Here are some key areas to focus on:

1. Data Model: Cosmos DB SQL and MongoDB have different data models, so it’s important to carefully analyze your data and design a data model that can efficiently and effectively store and retrieve your data in MongoDB. This can involve changes to your database schema, data types, and data relationships. In few cases this might be a optional step and can leverage the same schema.
2. Query Language: Cosmos DB SQL and MongoDB have different query languages, so you’ll need to update any inline queries in your application to work with MongoDB’s query language. You can also consider using a LINQ.
3. Indexing: MongoDB has a flexible indexing system that allows you to index fields in different ways. It’s important to consider the types of queries your application uses and create appropriate indexes to optimize performance.
4. Consider using a feature flag: Using a feature flag can help you test the performance and functionality of both the Cosmos DB SQL and MongoDB data access layers before completely switching to MongoDB. This can minimize any potential downtime or disruptions during the migration process.
5. Transactions: Cosmos DB SQL provides support for ACID (Atomicity, Consistency, Isolation, and Durability) transactions, whereas MongoDB guarantees ACID for operations on a single document. You’ll need to consider whether your application requires transaction support before adding transacation support directly.
6. Custom Converters: When migrating data from Cosmos DB to MongoDB, you may need to convert the data from JSON format to BSON format, since Cosmos DB uses JSON format for data storage and MongoDB uses BSON format.
7. Security: MongoDB provides a range of security features, including authentication, encryption, and access control. You’ll need to consider how to implement these features in your application to ensure the security of your data.

In the next section we will take a detailed look about three aspects

1. Inline query to LINQ query
2. Use Feature Flags to enable running two data access layers( CosmosDB & MongoDB) and disable anyone as needed.
3. Custom Converters : Convert all Custom JsonConverter classes to BsonConverter.

LINQ

Language-Integrated Query (LINQ) is a powerful.NET feature that provides a unified syntax for querying data from various sources, such as databases, collections, and XML documents.

LINQ is implemented as a set of extension methods on the IEnumerable & IQueryable interfaces, which allow you to chain together a series of operations to filter, sort, group, and transform data. The LINQ syntax is similar to SQL, making it easy to write and understand queries.

Here are some common LINQ operations:

• Filtering: You can use the Where method to filter data based on a specific condition. For example, the following code filters a list of numbers to only include even numbers:

var numbers = new List<int> { 1, 2, 3, 4, 5, 6 };
var evenNumbers = numbers.Where(n => n % 2 == 0);

• Projection: You can use the Select method to project the data into a different form. For example, the following code selects the names of all the employees in a list:

var employees = new List<Employee> { /* list of employees */ };
var employeeNames = employees.Select(e => e.Name);

• Joining: You can use the Join method to join data from two different sources based on a common key. For example, the following code joins a list of customers and orders on the CustomerId field:

var customers = new List<Customer> { /* list of customers */ };
var orders = new List<Order> { /* list of orders */ };
var customerOrders = customers.Join(orders, c => c.Id, o => o.CustomerId, (c, o) => new { c.Name, o.OrderDate });

• Grouping: You can use the GroupBy method to group data based on a specific key. For example, the following code groups a list of orders by customer:

var orders = new List<Order> { /* list of orders */ };
var ordersByCustomer = orders.GroupBy(o => o.CustomerId);

These are just a few examples of the many operations you can perform with LINQ. Because of its flexibility and expressiveness, it is a powerful tool for working with data in.NET.

LINQ + MongoDB

When working with MongoDB, you can use LINQ to write queries against your data by using a compatible LINQ provider. MongoDB has an official C#/.NET driver that supports LINQ queries using the Linq3 provider, starting with version 2.14.0.

Using LINQ with MongoDB can help you write more efficient, concise, and readable code. Below are some scenarios where you might want to use LINQ with MongoDB

1. Querying MongoDB collections: You can use LINQ to query MongoDB collections and retrieve data based on specific conditions. This can make it easier to work with complex queries and ensure that your queries are written in a type-safe and readable way.
2. Combining queries and aggregations: You can use LINQ to combine queries and aggregations together to build more complex queries against MongoDB. This can be useful when working with larger datasets or when you need to retrieve and analyze data from multiple collections.
3. Simplifying data transformations: LINQ provides a powerful set of operators that can be used to transform data, such as Select, Where, GroupBy, and OrderBy. You can use these operators to project data into a different format, filter data, group data, or sort data.
4. Writing more readable code: By using LINQ, you can often write code that is more concise and more readable than equivalent code written using traditional loops and conditionals.

Converting Inline Query in Code to LINQ

Below code is a low-level Find method to retrieve documents:

using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Driver.Linq;
using MongoDBLinq;

// creates a MongoClientSettings object using a connection string 
// that defines how to connect to the MongoDB server.
MongoClientSettings settings = MongoClientSettings.FromConnectionString("connectionString");

// creates a MongoClient instance with the MongoClientSettings object.
MongoClient client = new MongoClient(settings);

// gets a reference to a collection of Movie documents in a database 
// called "sample_mflix" by calling the GetDatabase and 
// GetCollection methods on the MongoClient instance
IMongoCollection<Movie> moviesCollection = client.GetDatabase("sample_mflix")
                                                 .GetCollection<Movie>("movies");


// creates a BsonDocument filter that specifies which 
// movies to retrieve based on their "year" property value 
// being between 1980 and 1990
BsonDocument filter = new BsonDocument{
    {
        "year", new BsonDocument{
            { "$gt", 1980 },
            { "$lt", 1990 }
        }
    }
};

// call the Find method on the IMongoCollection<Movie> instance 
// with the BsonDocument filter to retrieve a list of movies 
// that match the criteria specified in the filter.
List<Movie> movies = moviesCollection.Find(filter).ToList();

foreach (Movie movie in movies) {
    Console.WriteLine($"{movie.Title}: {movie.Plot}");
}

Equivalent LINQ Query

In this LINQ query, we use the AsQueryable method to convert the IMongoCollection<Movie> to a IQueryable<Movie>. Then we use the LINQ where clause to filter the movies based on their year. Finally, we use the select clause to project the movies and call ToList to execute the query and get the results as a list.

using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Driver.Linq;
using MongoDBLinq;

// creates a MongoClientSettings object using a connection string 
// that defines how to connect to the MongoDB server.
MongoClientSettings settings = MongoClientSettings.FromConnectionString("connectionString");

// sets the LinqProvider to V3. 
// This is used to enable the use of LINQ syntax to query the MongoDB database.
settings.LinqProvider = LinqProvider.V3;


// creates a MongoClient instance with the MongoClientSettings object.
MongoClient client = new MongoClient(settings);

// gets a reference to a collection of Movie documents in a database 
// called "sample_mflix" by calling the GetDatabase and 
// GetCollection methods on the MongoClient instance
IMongoCollection<Movie> moviesCollection = client.GetDatabase("sample_mflix")
                                                 .GetCollection<Movie>("movies");

IMongoQueryable<Movie> movies =
            from movie in moviesCollection.AsQueryable()
            where movie.Year > 1980 && movie.Year < 1990
            select movie;


foreach (Movie movie in movies) {
    Console.WriteLine($"{movie.Title}: {movie.Plot}");
}

Similarly identify all the similar inline queries and convert them to LINQ queries.

It’s worth noting that there may be cases where inline queries are more appropriate than LINQ, particularly for complex queries involving multiple collections or complex joins. In those cases, it may be preferable to continue using inline queries rather than forcing them into a LINQ syntax

.Net Analyzer for MongoDB

The .NET Analyzer for MongoDB is a tool that helps you write more efficient and effective MongoDB queries by analyzing your C# code and providing helpful suggestions and warnings. It can be used in conjunction with the MongoDB .NET driver and the Linq3 provider to create and optimize queries.

To install the package using the .NET CLI, open a command prompt and run the following command: dotnet add package MongoDB.Analyzer

Some of the features of the .NET Analyzer for MongoDB include:

1. Query translation: The analyzer can help you translate C# LINQ queries into their equivalent MongoDB queries, so that you can see how your code is being executed at the database level.
2. Query optimization: The analyzer can provide suggestions for optimizing your queries to make them faster and more efficient, such as using appropriate indexing or reducing the number of round trips to the server.
3. Type safety: The analyzer can help you avoid common type conversion errors by ensuring that your C# code matches the data types used in your MongoDB database.
4. Integration with Visual Studio: The analyzer integrates with Visual Studio to provide helpful warnings and suggestions as you write code, making it easier to catch potential issues early in the development process.

FeatureFlag

Feature flags, also known as feature toggles, are a powerful technique for managing software releases and enabling experimentation. In .NET, there are several options for implementing feature flags.

One popular open-source library for feature flags in .NET is FeatureToggle by Jason Roberts. This library provides a simple API for defining and evaluating feature flags and includes support for toggling features based on a variety of conditions, such as user roles, environment variables, and date and time ranges.

Another option is to use Microsoft’s Azure App Configuration service, which includes built-in support for feature flags and integrates with.NET applications through the Microsoft.Extensions.Configuration library. This provides a centralized location for managing feature flags and other application configuration settings, and includes support for gradual rollouts, targeted rollouts, and A/B testing.

FeatureFlag to switch between Data Access Layer

Now lets see how to use Feature Flags in an ASP.NET Core application to switch between two data access layers for CosmosDB and MongoDB:

First, let’s assume that you have two separate implementations of your data access layer, one for CosmosDB and one for MongoDB. You can define an interface for your data access layer like this:

public interface IDataAccessLayer
{
    Task<IEnumerable<MyData>> GetData();
}

Create separate implementations of the data access layer for CosmosDB and MongoDB:

public class CosmosDbDataAccessLayer : IDataAccessLayer
{
    // Implementation for CosmosDB data access
}

public class MongoDbDataAccessLayer : IDataAccessLayer
{
    // Implementation for MongoDB data access
}

Next, you can use Feature Flags to switch between the two implementations based on a configuration value in the Startup class:

public void ConfigureServices(IServiceCollection services)
{
    // Register the Feature Flags service
    services.AddFeatureFlags(Configuration.GetSection("FeatureFlags"));

    // Register the data access layer based on the Feature Flag
    if (features.IsEnabled("UseCosmosDb"))
    {
        services.AddSingleton<IDataAccessLayer, CosmosDbDataAccessLayer>();
    }
    else
    {
        services.AddSingleton<IDataAccessLayer, MongoDbDataAccessLayer>();
    }
}

We’re using the AddFeatureFlags method to register the Feature Flags service, which reads the configuration from the appsettings.json file. Then, we’re using the IsEnabled method to check the value of the “UseCosmosDb” Feature Flag. If the flag is enabled, we register the CosmosDB data access layer, and if it’s disabled, we register the MongoDB data access layer.

Finally, you can use the IDataAccessLayer interface in your controller or other services:

public class MyController : Controller
{
    private readonly IDataAccessLayer _dataAccessLayer;

    public MyController(IDataAccessLayer dataAccessLayer)
    {
        _dataAccessLayer = dataAccessLayer;
    }

    public async Task<IActionResult> Index()
    {
        var data = await _dataAccessLayer.GetData();
        // ...
    }
}

This controller will receive an instance of the data access layer based on the Feature Flag that’s currently enabled.

With this approach, you can easily switch between two different data access layers using Feature Flags, without having to change any code or configuration files.

Custom Converters

When migrating data from Cosmos DB to MongoDB, you may need to convert your custom converter classes from JsonConverter to BsonConverter so that they can be used to serialize and deserialize BSON documents instead of JSON documents.

BsonConverter

BsonConverter is a class in the MongoDB C# driver that provides functionality for converting .NET objects to and from BSON (Binary JSON) format. BSON is a binary serialization format used by MongoDB to store and exchange data.

Create an instance of the BsonConverter class and use its methods to serialize or deserialize .NET objects to and from BSON format.

Here’s an example of how you might convert the TransactionDocumentJsonConverter to a TransactionDocumentBsonConverter

using MongoDB.Bson;
using MongoDB.Bson.IO;
using MongoDB.Bson.Serialization;
using MongoDB.Bson.Serialization.Serializers;

public class TransactionDocumentBsonConverter : BsonConverter
{
    public override Type ValueType => typeof(TransactionDocument);

    public override object Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
    {
        var bsonReader = context.Reader;

        bsonReader.ReadStartDocument();

        string id = null;
        string transactionId = null;
        int amount = 0;

        while (bsonReader.ReadBsonType() != BsonType.EndOfDocument)
        {
            var elementName = bsonReader.ReadName();

            switch (elementName)
            {
                case "_id":
                    id = bsonReader.ReadString();
                    break;

                case "transactionId":
                    transactionId = bsonReader.ReadString();
                    break;

                case "amount":
                    amount = bsonReader.ReadInt32();
                    break;

                default:
                    bsonReader.SkipValue();
                    break;
            }
        }

        bsonReader.ReadEndDocument();

        var doc = new TransactionDocument
        {
            Id = id,
            TransactionId = transactionId,
            Amount = amount
        };

        return doc;
    }

    public override void Serialize(BsonSerializationContext context, BsonSerializationArgs args, object value)
    {
        var doc = (TransactionDocument)value;
        var bsonWriter = context.Writer;

        bsonWriter.WriteStartDocument();

        bsonWriter.WriteName("_id");
        bsonWriter.WriteString(doc.Id);

        bsonWriter.WriteName("transactionId");
        bsonWriter.WriteString(doc.TransactionId);

        bsonWriter.WriteName("amount");
        bsonWriter.WriteInt32(doc.Amount);

        bsonWriter.WriteEndDocument();
    }
}

In this example, the TransactionDocumentBsonConverter class extends the BsonConverter class instead of the JsonConverter class.

The Deserialize() and Serialize() methods are overridden to provide custom deserialization and serialization logic for TransactionDocument objects. The ValueType property is overridden to indicate that this converter should be used for objects of type TransactionDocument.

In the Deserialize() method, a new instance of BsonReader is created by accessing the BsonDeserializationContext.Reader property. The method then reads the BSON data from the BsonReader, extracts the values of the properties of the TransactionDocument, creates a new instance of TransactionDocument, and sets its properties using the values extracted from the BsonReader.

In the Serialize() method, a new instance of BsonWriter is created by accessing the BsonSerializationContext.Writer property. The method then writes the properties of the TransactionDocument to the BsonWriter. The method takes the TransactionDocument object as an input parameter and writes its properties to the output stream.

Below is an example of TransactionDocumentJsonConverter before conversion.

using Newtonsoft.Json;
using Newtonsoft.Json.Linq;

public class TransactionDocumentJsonConverter : JsonConverter
{
    public override bool CanConvert(Type objectType)
    {
        return objectType == typeof(TransactionDocument);
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        JObject obj = JObject.Load(reader);

        // Read the properties of the TransactionDocument
        string id = (string)obj["_id"];
        string transactionId = (string)obj["transactionId"];
        int amount = (int)obj["amount"];

        // Create a new TransactionDocument instance and set its properties
        TransactionDocument doc = new TransactionDocument();
        doc.Id = id;
        doc.TransactionId = transactionId;
        doc.Amount = amount;

        return doc;
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        TransactionDocument doc = (TransactionDocument)value;

        JObject obj = new JObject();

        // Write the properties of the TransactionDocument
        obj["_id"] = doc.Id;
        obj["transactionId"] = doc.TransactionId;
        obj["amount"] = doc.Amount;

        obj.WriteTo(writer);
    }
}

Conclusion

We looked at some of the most important factors to consider when migrating an application from CosmosDB SQL to MongoDB. Other aspects may be overlooked, and the decision to adapt the aforementioned aspects will be based on your specific needs and requirements.

That would be the end of this series.

Happy Migration !!!