CloudnRoll Blog

  • Participation in the Veeam Community Weekly Recap number 161

    Participation in the Veeam Community Weekly Recap number 161

    The Veeam Community Weekly Recap is available with incredible content and relevant information! Congratulations to Nico Losschaert, Brad Linch, Christian Eromosele, and Philippe Dupuis for the posts and articles.

    Thanks so much to Madalina Cristil, Mark Boothman, and Safiya Mohamed for the opportunity to participate! I am honored.

    Link:
    Back Up Audit, Hidden Cloud Costs and Active Directory Rename | Veeam Community Resource Hub


  • Implementing Snapshot Orchestration for Critical Environments

    Implementing Snapshot Orchestration for Critical Environments

    Snapshot orchestration is critical, especially in IT environments that seek agility in application protection and recovery operations. The capability for fast restoration from snapshots also helps reduce downtime and minimizes business impact.

    Veeam Backup & Replication can integrate with storage systems and create backups using storage-based snapshots, minimizing production impact and accelerating the backup process.

    In addition to the traditional Backup Job, Veeam offers the Snapshot Orchestration Job. It automates the creation, management, and deletion of snapshots, ensuring consistent backups, minimal impact on production systems, and more efficient storage use.

    Together, the Veeam Backup and Veeam Snapshot Orchestration enable the implementation of a comprehensive data protection and recovery strategy.

    vSphere Metro Storage Cluster (vMSC)

    In critical VMware environments that require high availability, VMware High Availability (HA) is a technology that provides cluster-based monitoring of virtual machines running on included ESXi hosts. If the storage, network, or host fails, the remaining ESXi hosts coordinate to restart the affected virtual machines on unaffected hosts.

    VMware HA is made possible via shared storage. Without shared storage, VMs and their data cannot be seen by surviving hosts, and therefore a disaster restart operation is not an option.

    VMware vSphere Metro Storage Cluster (vMSC) is a configuration designed to enable high availability, disaster recovery, and fault tolerance for virtualized workloads by integrating VMware vSphere with a storage solution that supports synchronous replication across two geographically separated data centers.

    When combined with vSphere’s HA features, vMSC ensures that virtual machines (VMs) running on one site can quickly be restarted on the other site in case of failure without losing data.

    vMSC is also fully compatible with vSphere DRS (Distributed Resource Scheduler) features, which help ensure that virtual machines are automatically recovered and resources are balanced across sites.

    Additionally, vMSC uses synchronous storage replication to keep data mirrored in real-time between two storage systems, usually in separate data centers (metro distances). This ensures that both sites have an identical copy of the data, providing fault tolerance and minimizing downtime.

    Pure Storage Purity ActiveCluster

    Purity ActiveCluster is a feature of FlashArrays that enables two-site synchronous replication for high availability. It is a fully symmetric, active/active bidirectional replication solution that provides synchronous replication for RPO zero and automatic transparent failover for RTO zero. VMware virtual machines can access storage in both sites simultaneously without clustering multiple sites, enabling clustered arrays and hosts to deploy flexible active/active data center configurations.

    When paired with VMware’s vSphere Metro Storage Cluster (vMSC) configuration, VMware virtual machines can access storage in both sites simultaneously, without interruption, even if one site fails.

    On the other hand, when Purity ActiveCluster is combined with VMware High Availability (HA), automatic failover across sites ensures that VMs maintain high availability even if a data center goes down.

    Additionally, Pure Storage’s synchronous replication allows data to be consistently shared between two different sites, but the storage access technology can be configured differently at each location. It exposes paths to local hosts as active/optimized paths and exposes paths to remote hosts as active/non-optimized.

    The optimized path is defined on a per host-to-volume connection basis using a preferred array option; this ensures that regardless of what host a VM or application is running on, it will have a local optimized path to that volume, as presented below.

    The Purity ActiveCluster consists of three core components: thePure1 Mediator, active/active clustered array pairs, and stretched storage containers.

    Some key features include:

    • Integrated Pure1 Cloud Mediator: An automatically configured passive mediator that enables transparent failover and prevents split-brain scenarios, eliminating the need to deploy and manage additional components.
    • Active/Active Clustered FlashArrays: Utilize synchronous replication to maintain identical copies of data across both arrays and present them as a consistent copy to hosts connected to either or both arrays.
    • Stretched Storage Containers: Management containers that group storage objects, such as volumes, and stretch them across two arrays.
    • Synchronous Replication: Writes are synchronized between arrays and stored in non-volatile RAM (NVRAM) on both arrays before being acknowledged to the host.
    • Symmetric Active/Active: This option allows read and write operations on the same volumes on either side of the mirror, with optional host-to-array site awareness.
    • Transparent Failover: Automatic, non-disruptive failover between synchronously replicating arrays and sites, including automatic resynchronization and recovery.
    • No Bolt-ons & No Licenses: No additional hardware or costly software licenses are required—simply upgrade the Purity Operating Environment to enable active functionality.
    • Simple Management: This feature allows data management operations, such as provisioning storage, connecting hosts, creating snapshots, and cloning, to be performed from either side of the mirror.

    Snapshot Orchestration: Pure Storage FlashArray and Veeam Data Platform V12

    The previous context is crucial for understanding how Veeam Backup & Replication v12 or later versions can orchestrate snapshots of Pure FlashArrays configured in ActiveCluster mode.

    Synchronous replication in ActiveCluster mode provides a symmetric, active-active solution that enables volume mirroring at the storage layer. A volume or storage snapshot created on either array is synchronously replicated across both arrays, ensuring data consistency.

    With the release of the Veeam Backup & Replication v12, Veeam has enhanced the functionality of its USAPI v2. This new API version supports the orchestration of replicated snapshots and enables the offloading or archiving of storage snapshots in their native snapshot format to secondary storage.

    When integrated with Veeam Backup & Replication via USAPI v2, Purity ActiveCluster’s primary use case is to leverage a Veeam Snapshot-Only Job to create a synchronous storage snapshot on both the local and secondary FlashArrays.

    This snapshot is then used as the source for a Veeam Backup Job.

    In addition, Veeam’s Application-Aware Image Processing ensures application-consistent backups. This ensures that the snapshot captures all necessary data, allowing for the recovery of applications to a consistent state without corruption or data loss.

    The most important benefit is that storage snapshots allow you to accelerate backup and replication for VMware vSphere VMs with disks hosted on storage systems, reducing the VMware VM snapshot lifetime and minimizing the impact on applications.

    Configuring Snapshot-Only Jobs + Backup Jobs from Snapshots

    Veeam has released a highly informative document titled “Advanced Storage Snapshot Integration with Pure Storage FlashArray and Veeam Platform V12.”

    The document effectively illustrates the scenario described earlier: a Veeam job that orchestrates the creation of storage snapshots on the remote Pure FlashArray, followed by the creation of a Veeam backup job using the storage snapshot from the remote Pure FlashArray in ActiveCluster mode.

    Below, the job action panel shows that the snapshot-only for synchronous replicated VM was successfully created and executed. Highlighted in red, we can see that Veeam is aware of the synchronous replication relationship; the snapshots were made simultaneously on both the local and remote Pure FlashArrays and the backup was created from the snapshot.

    You can access the document here on this link:

    https://www.veeam.com/resources/wp-pure-storage-veeam-advanced-storage-snapshot.html

    Conclusion

    Veeam Backup & Replication v12 and later versions provide a robust solution for orchestrating and managing storage snapshots in environments utilizing Pure FlashArrays in ActiveCluster mode. In this scenario, Veeam ensures consistent snapshot creation across local and remote arrays by leveraging synchronous replication, minimizing the impact on production systems while enhancing backup and replication efficiency.

    Integrating Veeam’s Snapshot-Only Jobs and Application-Aware Image Processing guarantees application consistency during backups, ensuring reliable and efficient data protection. Furthermore, offloading and archiving storage snapshots to secondary storage simplifies backup operations, offering a scalable and flexible solution for critical VMware vSphere environments.

    Overall, this integration accelerates backup processes, reduces downtime, and ensures data integrity, making it an essential tool for businesses that require dependable disaster recovery and high availability.

    References

    https://support.purestorage.com/bundle/m_user_guides_for_vmware_solutions/page/Solutions/VMware_Platform_Guide/User_Guides_for_VMware_Solutions/ActiveCluster_with_VMware_User_Guide/topics/concept/c_activecluster_introduction.html

    https://helpcenter.veeam.com/docs/backup/storage/snapshot_only_job_perform.html?ver=120

    https://helpcenter.veeam.com/docs/backup/storage/backup_from_storage_snapshots.html?ver=120

  • Highlights from the Veeam 100 Summit 2024

    Highlights from the Veeam 100 Summit 2024

     

    I would like to share the content presented during the Veeam 100 Summit in Berlin, which was brilliantly organized by the Veeam team in a highly professional manner. I would also like to thank the following individuals for inviting me to the Summit:

    • Rick Vanover, Office of the CTO – Vice President Product Strategy;
    • Nikola Pejková, Veeam Vanguard Community Manager & Technical Analyst;
    • Madalina Cristil, Principal Community Manager at Product Strategy-Office of the CTO-Veeam Software;
    • Safiya Mohamed, Community Analyst.

    All the hard work and enormous effort involved in planning, implementing, and hosting the Veeam 100 Summit can be better understood and appreciated in this excellent article by Nikola Pejková:

    Veeam100 Summit: The Power of Veeam100 Community

    When I arrived at the hotel on Sunday afternoon, I saw that all the Veeam team’s preparations for the next day had been completed! Once again, congratulations to the organization team!

     

    I finally had the opportunity to meet the Veeam Community and Veeam Vanguard teams in person. I greatly appreciate Madalina Cristil, Safiya Mohamed, and Nikola Pejková’s warm and respectful welcome.

    On Monday morning, I received my swag. It was my first time as a Vanguard, and I received a lovely package that included a beautiful backpack, a Veeam Vanguard jacket, a thank-you card, and a trophy!

    The trophy design was created exclusively for Veeam Vanguards and Legends. It is made from unbreakable, eco-friendly material and personalized for every program member!

     

    Workshop: The BCDR Survivalist Guide for Veeam Recovery Orchestrator

    The first event took place on Monday afternoon. It featured a Veeam Recovery Orchestrator workshop and hands-on labs led by Emilee Telez and the VRO team. In addition to the laboratory activities, advanced VRO capabilities—such as recovery to Microsoft Azure from Veeam Agent and vSphere VM backups created by Veeam Backup & Replication—were showcased.

     

     

     

    Program Intro & Welcome

    Rick Vanover, Madalina Cristil, Nikola Pejková, Safiya Mohamed, and Hannes Kasparick effectively opened the Summit on Tuesday morning.

    General guidelines and an overview of the Veeam 100 program were presented, including the growth in the number of members and the presence of VUG leaders at the event. Rick also explained the program’s priorities regarding products and services for the community.

    The results of the 2024 Veeam Hackathon and a summary of global events from the various Veeam User Groups were also presented. The session ended by introducing new planned and ongoing projects and the local MVP representatives.

     

     

    Sponsors Sessions: Wasabi and Object First

    Drew Schlussel and Rob Callaghan presented Wasabi’s main differentiators, new features, and the planned product roadmap.

    Sterling Wilson, representing Object First, presented the ZTDR (Zero Trust Data Resilience) concept, integration with the Veeam platform through the Smart Object Storage API, and the new sizing tool.

     

     

    Veeam Data Platform Core Updates

    Anton Gostev and the Veeam Product Management team presented the new features in Veeam Backup & Replication release 12.3 and version 13. As some content cannot be shared now, I’m sharing images of the significant features of release 12.3 that are already public. I plan to write specific articles about release 12.3 and VBR’s new version 13 soon.

     

     

     

     

     

    Red and Yellow Content Sessions: Veeam Data Cloud Near-Term Updates and Coveware by Veeam: Real-World Observations From the Field

    Unfortunately, due to the red and yellow content of these sessions, I cannot comment further on these topics. Stay tuned, and I’ll share more news soon!

     

    On the third day, the sessions were divided into two rooms. Attendees could choose the session they wanted to attend, although it was difficult because they were all great. Below are the sessions I attended:

    VCSP Updates in Veeam Data Platform and Veeam Data Cloud

    As this session has been classified as yellow and red content, I cannot share more information at this time. Let’s wait for the news to come!

     

    Veeam Data Cloud Futures

    In this session, classified as red content, the Veeam PM team shared the plans and roadmap for Veeam Data Cloud services with those present.

    Product Update: Veeam Kasten for Kubernetes

    This session, led by Michael Kade, presented the status of the Kubernetes project after 10 years, the concept of Kubernetes Operators, the incorporation of AI/ML in Kubernetes environments, the adoption of VMs on Kubernetes, the integration of Kasten with KuberVirt (Red Hat OpenShift and Rancher+ Harvester), and the new features in Veeam Kasten’s version 7.5.

     

     

    Upleveling the conversation from B/R to IT DR to CR to BC/CM and Beyond

    This was an excellent session led by Jason Buffington, offering valuable insights on business resilience from the perspective of organizations and professionals. The session also covered how to translate ‘Cyber Resilience’ and ‘Disaster Recovery’ for those who don’t speak ‘backup,’ as well as the five key conversations for BC/CM/BR stakeholders.

     

    Yellow content Sessions: Veeam Alliances – Security and Storage Integrations Update and Veeam Backup for Public Cloud Updates

    On Wednesday afternoon, I had the opportunity to participate in two sessions.

    In the first session, Veeam presented its partnership strategies with storage and cybersecurity solution providers. In the second session, Veeam discussed its plans to integrate the platform with public cloud providers and services.

    As both sessions were classified as yellow content, I cannot share further details now. However, I plan to write about these topics soon on my blog and in the Veeam community.

     

    On the last day of the Veeam 100 Summit, the traditional AMA session with Veeam’s Product Management team was held and classified entirely as red content.

    There was much interaction between the participants and the Veeam PM team, with members of the Veeam 100 community answering many questions and clarifying doubts. Soon after, the event ended.

     

    Thank you to everyone at Veeam, including Rick Vanover, Nikola Pejková, Safiya Mohamed, and Madalina Cristil, for being amazing supporters throughout the year and leading up to this great event.

    I also wrote about my extra-event experience in Berlin, during which I had the opportunity to interact with iconic places and remember relevant facts from history.

    You can access this write-up here: Veeam 100 Berlin experience: revisiting history

    Check out posts from Veeam Vanguards and Veeam Legends attendees about the Summit:

     

  • My experience with the VMCA 2024 certification

    My experience with the VMCA 2024 certification

    The new Veeam Certified Architect certification format presents sizing and architecture scenarios, and I liked it, although it is more challenging than the previous format.

    I passed the exam in January 2024, shortly after Veeam made the exam available.

    “The VMCA 2024 demonstrates the knowledge and expertise required to design, deploy, and optimize data protection architecture with Veeam Backup & Replication. Holders of the VMCA 2024 can be trusted to design large-scale/enterprise deployments and manage multiple customers in on-site, off-site, and hybrid environments. The VMCA 2024 demonstrates a high level of knowledge.”

    I recommend carefully studying the official training material and carrying out the exam simulation provided by Veeam. Also, carefully study the architecture, sizing, and user guide documents that Veeam makes available.

    ✅ Official Training: https://www.veeam.com/vbr_ad_course_outline_ds.pdf

    ✅ Student Guide: https://www.veeam.com/veeam_certified_architect_v12_exam_guide_ds.pdf

    Experience using Veeam official documentation, like user guides and best practices, and designing Veeam architectures also dramatically helps.

    ✅ Veeam Help Center: https://www.veeam.com/support/help-center-technical-documentation.html

    ✅ Veeam Backup & Replication Best Practices: https://bp.veeam.com/vbr/

    The important thing is not to rush to study and be prepared: see the process as a journey to improve your knowledge.

    In the Veeam community, there are many testimonials and tips from members who have passed the exam and study groups.

    I recommend that you participate in these initiatives and read the community posts: many doubts clarified by the Veeam Legends and community members can be part of the exam.

    ✅ Veeam Community: https://community.veeam.com/search?q=vmca%202024

    I also recommend using the hands-on labs and resources at Veeam University!

    ✅ Hands-on labs: https://go.veeam.com/hands-on-lab-experience

    ✅ Veeam University: https://veeamuniversity.com/

    As a contribution, I am sharing a file with my annotations for each item described in the “Exam Outline” section of the Student Guide:

    ✅ https://www.dropbox.com/scl/fi/mk4vwd94eczgl13146ds7/VMCA2024-Exam-Outline-Notes_v1.pdf?rlkey=s8ms38fl14o8mqxi3zsq55569&st=j6xcyt0m&dl=0

    Good luck and good studies! The effort and dedication are worth it! 😀

  • Report: Top Trends in Tech for VARs and MSPs Companies

    Report: Top Trends in Tech for VARs and MSPs Companies

    If your company is a Value-Added Reseller (VAR) and/or a Managed Service Provider (MSP), I recommend reading the just released “Third Annual Direction of Technology Ecosystem Report” from TD Synnex.

    This survey was conducted with +1000 Channel Partners from April to June 2024. Five main channel partners´ trends were revealed, and I want to highlight some interesting insights:

    ✅ Channels are prioritizing specific professional skills in their hiring processes: consulting (43%) and managed services (37%).

    ✅ They say the top four solutions they plan to offer include: security (49%), AI/GenAI (44%), hybrid cloud (39%), and networking (38%).

    ✅ AI is reshaping the channel´s offerings; in the next 12 months, more than half expect to offer GenAI (52%) and AI Services (48%). They plan to provide AI/GenAI from 14% in 2022 to 44% in 2024, a 214% increase.

    IMO, it´s clear that providing AI and GenAI solutions and services is essential for VARs and MSPs as these technologies enhance their value proposition, boost operational efficiency, facilitate scalability, and provide more added value in their offerings and services.

    By incorporating AI, they can deliver personalized solutions that enhance decision-making and customer satisfaction, establishing themselves as strategic partners/trusted advisors in their customer´s business transformation efforts.

    For reference, I can share some examples of AI and GenAI solutions/services:
    ✔ Chatbots and Virtual Assistants;
    ✔ Cloud-based AI Platforms;
    ✔ AiOps Solutions;
    ✔ Predictive Analytics and Automated Incident Response;
    ✔ Automated Document Processing;
    ✔ Natural Language Processing (NLP);
    ✔ GPU + GPUaaS;
    ✔ AI Infused Data Lakes;
    ✔ DevSecOps GenAI tools – software development, testing, deployment;
    ✔ SRE and Observability tools;
    ✔ Robotic Process Automation (RPA);
    ✔ Personalized Marketing Solutions;
    ✔ Business Continuity and Disaster Recovery (BCDR).

    To access the report, click on: https://content.tdsynnex.com/dot/files/Third-Annual-Direction-of-Technology-Full-Report.pdf

  • The DevOps Research and Assessment (DORA) report

    The DevOps Research and Assessment (DORA) report

    The 10th DORA annual report was released yesterday. DORA stands for the DevOps Research and Assessment team, which is part of Google Cloud.

    Generative AI is reshaping software development practices, and most development professionals are relying on AI to help them perform their core role responsibilities and increase productivity.

    To access the report, click on: https://cloud.google.com/resources/devops/state-of-devops

    You can also join the DORA Community: https://dora.community/

  • C.A.R.E. Framework: Boost Machine Learning Success

    C.A.R.E. Framework: Boost Machine Learning Success

    Data Quality C.A.R.E. is a framework that emphasizes essential aspects of data quality, particularly in the context of Machine learning (ML) and Deep Learning (DL).

    Before we describe the framework, it is important to remember essential concepts.

    Machine Learning is a subset of Artificial Intelligence (AI) that enables systems to learn from data and improve their performance over time without being explicitly programmed.

    It is generally effective with smaller datasets and can perform well with a few hundred or thousand samples. It is commonly used for structured data tasks like fraud detection, customer segmentation, and predictive analytics.

    Deep Learning is a specialized subset of Machine Learning that uses neural networks with many layers (deep neural networks) to model complex patterns in data. It requires large volumes of data to train effectively, often needing millions of samples to avoid overfitting and to capture complex patterns.

    Deep Learning excels in unstructured data tasks such as image recognition, natural language processing, and speech recognition, where it can outperform traditional ML methods.

    L.L.M.s (Large Language Models) are specialized deep-learning models tailored to language tasks. Classic examples are GPT-4,  Gemini, and Claude.

    Adopting a Data Quality Framework is crucial for Machine Learning (ML) and Deep Learning (DL) initiative success in organizations:

    • The accuracy and reliability of ML and DL models heavily depend on the data quality used for training. A robust data quality framework ensures the data is complete, accurate, and relevant.
    • High-quality data helps models learn the correct patterns, avoiding producing biased or incorrect predictions undermining their utility.
    • A structured approach to data quality helps identify and eliminate noise and inconsistencies in the data.
    • Data preparation and cleaning can be time-consuming and costly in the ML/DL pipeline.
    • Reliable data leads to accurate analyses, which are essential for informed decision-making.
    • Maintaining high data quality is vital for compliance, data governance, and privacy regulations.
    • A solid data quality framework allows for scalable data quality management.

    Source:

    The Data Quality Framework C.A.R.E. is a structured approach designed to assess and enhance data quality within organizations.

    The acronym C.A.R.E. represents four essential dimensions of data quality:

    Connected

    The data is connected if it has no gaps; all required data is present.   

    Disconnected data can lead to biased insights and poor outcomes. It is also harder to handle because most algorithms cannot handle them. If records containing missing data are discarded or considered, it leads to assumptions and biases.

    Accurate

    Accuracy measures how closely data values reflect the real-world entities or conditions. It assesses the correctness of the data; there are little or no measurement errors.

    Inaccurate data may silently break a model since values wildly outside the expected range may skew the model, impacting performance or producing absurd predictions. It can lead to losing trust in the model and potentially costly errors. Ensuring accuracy builds confidence in the data and its derived conclusions.

    Relevant

    The data is relevant if it is related to the task or the phenomenon context. This dimension focuses on ensuring that the data meets the users’ needs.

    Using relevant data helps minimize distractions from unnecessary information, allowing organizations to focus on what truly matters for their analyses and decisions. Relevant data enhances the effectiveness of insights drawn from analyses.

    Enough

    It is necessary to define the amount of data depending on the model’s complexity and the associated costs to store and process it.

    Enough data management practices lead to cost savings and streamlined operations.

    Organizations can enhance productivity, reduce processing times, and focus more on analysis than data handling by optimizing data processes.

    It is also possible to reduce costs using data augmentation, which incorporates more data in a dataset to train a model without collecting more data.

    In conclusion, adopting a data quality framework like C.A.R.E is essential for successfully deploying Machine Learning and Deep Learning initiatives with good results and saving costs. It is critical to avoid data distrust – a lack of confidence or skepticism regarding data quality, integrity, or reliability.

    References:

    https://www.gartner.com/smarterwithgartner/how-to-improve-your-data-quality

    https://training.linuxfoundation.org/training/pytorch-and-deep-learning-for-decision-makers-lfs116

  • The Veeam Calculators v1.1 are available

    The Veeam Calculators v1.1 are available

    If you are sizing your Veeam architecture, it is important to consult the new official version of the Veeam calculators.

    Please, refer to the most recent announcement in Veeam Community:

    https://community.veeam.com/calculator-commons-151/calculators-1-1-released-8220?postid=65024#post65024

    Link to the official Veeam Calculators:

    https://www.veeam.com/calculators

  • Microsoft VSS Framework: Recap and Troubleshooting

    Microsoft VSS Framework: Recap and Troubleshooting

    Sometimes, we need to remember some basic concepts to understand what happens “under the hood” in operations we perform on a day-to-day basis, such as performing an application-consistent backup and how to troubleshoot it.

    So, what is Microsoft VSS?

    Microsoft Volume Shadow Copy Service (VSS) is a volume backup framework in Microsoft Windows Operating Systems.

    It integrates with many applications and backup solutions, like Veeam Backup & Replication, to provide application-consistent backups.

    Why is VSS used?

    Some enterprise applications maintain transactional interdependencies between data files, logs, metadata, and in-memory data.

    For these applications, an application-consistent backup ensures that all these transactional elements are synchronized when captured by the backup request

    Here, we show examples of applications integration with Microsoft VSS for consistent backups:

    • Databases: SQL Server, Oracle, MySQL;
    • Email Server: Microsoft Exchange;
    • Enterprise Applications: SAP, Microsoft Dynamics;
    • Hypervisors: Microsoft Hyper-V, VMware ESXi (via VMware Tools), Nutanix AHV (via NGT), etc;
    • File Systems: Windows File Server and Windows Distributed File Systems.

    VSS Components

    A VSS backup solution consists of the following components:

    • The VSS Service is part of the Windows Operational System and is responsible for the communication between all other components
    • The VSS Requestor is typically a backup application that requests snapshots. It can be, for example, a Windows Agent from Veeam Backup & Replication.
    • The VSS Writer is an application component that ensures a consistent data set before taking the snapshot.
    • The VSS Provider is in charge of creating and maintaining the snapshots. A provider can be based on software or hardware.

    How does VSS work?

    The diagram at the top of the page shows the interaction between all VSS components when processing a backup request.

    Summarizing the processes, we have:

    • Veeam Backup & Replication (or any VSS Requestor) initiates a backup request for the VSS Service.
    • After the backup initialization step is concluded, the writer enters a Quiescence phase, ensuring that all pending transactions are completed and all in-memory data is synchronized with the data on disk.
    • After that, VBR requests the snapshot, and the Writers enter a Freeze state. The application writes are held, and the caches are flushed to disk.
    • After the Provider commits the snapshot, the Writer “thaws” the application to write I/O requests. Observation: the application freeze cannot take longer than sixty seconds.
    • Veeam VBR imports the snapshot, and the backup operation is completed.
    • Veeam VBR triggers VSS Service to remove the persistent VSS snapshot, and the VSS backup process is finished.

    If a problem occurs in any part of this process, a troubleshooting action will be necessary.

    Troubleshooting Microsoft VSS

    First step: get familiar with the VSS Framework

    Microsoft VSS:

    https://learn.microsoft.com/en-us/windows-server/storage/file-server/volume-shadow-copy-service

    Application-Aware Processing:

    https://helpcenter.veeam.com/docs/backup/vsphere/application_aware_processing.html?ver=120

    The second step is to check for Veeam Support Knowledge Base (KBs)

    Verify if it is a Know Challenge and apply the Know Solution. If the issue is not listed in KBs and persists, proceed with the third step (manual troubleshooting).

    KB 1789: How to Locate and Collect VSS/VIX Log Files From Guest OS

    https://www.veeam.com/kb1789

    KB 316: How to Use VSSTrace to Collect VSS Diagnostic Logging

    https://www.veeam.com/kb3164

    KB 1980: Using the Diskshadow Utility to Manually Test VSS Operations

    https://www.veeam.com/kb1980

    KB 2041:  Windows VSS Writers and Corresponding Service Names

    https://www.veeam.com/kb2041

    KB 2095: SQL VSS Writer is missing: databases will be backed up in crash-consistent state and transaction log processing will be skipped

    https://www.veeam.com/kb2095

    KB1855:  Hyper-V Guest processing skipped (check guest OS VSS state and integration components version)

    https://www.veeam.com/kb1855

    KB 1794: Insufficient Storage for VSS Error “0x8004231F or Error: VSSControl: -2147467259”

    https://www.veeam.com/kb1794

    KB 1697: VSS Errors Related to the ‘NTDS’ VSS Writer

    https://www.veeam.com/kb1697

    KB 1680: VSS Timeout when backing up Exchange VM

    https://www.veeam.com/kb1680

    KB 1377: VSS Freeze Wait Timeout

    https://www.veeam.com/kb1377

    KB 1785: 0x8004230F or Error: VSSControl: -2147212529 (VSS_E_UNEXPECTED_PROVIDER_ERROR)

    https://www.veeam.com/kb1785

    KB 4333:   Veeam’s VSS Operations With SMB 3.0 Impacted by Windows June 14, 2022 Update

    https://www.veeam.com/kb4333

    KB 4573: Backup fails with: “The system cannot find the file specified.” or “The device is not ready.”

    https://www.veeam.com/kb4573

    Third step: manual troubleshooting

    Verify the prerequisites:

    • Windows VSS Services should be running;
    • Volume with shadow storage has enough free space;
    • Ensure that VSS writers are enabled and in a consistent state;
    • At least one NTFS volume is present on Windows XP or older Windows systems.

    Verify for Veeam VBR errors and verify if any previous KB addresses it.

    If not, you can try to verify the Windows errors:

    • Run “eventvwr.msc” command;
    • Get Windows Logs for Applications and check errors associated with VSS. Search for solutions in Microsoft Support articles; for example, see this Microsoft TechNet Article;
    • Get Windows Logs System and check errors associated with VSS. Verify if it is a Know Chall and apply the Know Solution from Veeam KBs.

    If it does not work, try to verify VSS Errors:

    • Run “vssadmin list writers” command;
    • If a writer is in “retryable” error, restart the writer and retry the backup;
    • If the writer is still in an error state, validate it using Diskshadow as a VSS Provider. If a writer fails during this operation or creates an incomplete snapshot, troubleshoot the writer.  

    If all previous steps fail, collect all logs and open a case with the Veeam Support team to get help.

    I hope this article was helpful.

  • Veeam Backup for Microsoft 365 v8: Improved Architecture

    Veeam Backup for Microsoft 365 v8: Improved Architecture

    Veeam Backup for Microsoft 365 v8 has been out and generally available since September 3, 2024, with new features focused on complete immutability, increased scalability, and more.

    I highlight these changes in the architecture, as shown in the figure above.

    Let’s verify some of the new features.

    Comprehensive Immutability

    Veeam Backup for Microsoft 365 allows you to prevent data deletion from the primary object storage repository by making the data temporarily immutable, thus safeguarding it against malware threats.

    The following types of Object Storage repositories are compatible with this release and can support immutability:

    • S3 object storage: compatible with the AWS S3 operations and AWS S3 Signature Version 4 standard.
    • Amazon S3 object storage.
    • Microsoft Azure Blob storage.
    • IBM Cloud Object Storage.
    • Wasabi Cloud Object Storage.

    You can activate data immutability when adding a new object storage repository to Veeam Backup for Microsoft 365. It ensures that once Veeam Backup for Microsoft 365 stores data in the repository, it cannot be modified or deleted during the immutability period—whether that’s a specific number of days or the entire retention period of the repository—allowing you to customize your data protection strategies to fit your requirements.

    Data immutability offers two modes: compliance (default) and governance. You can enable governance mode when configuring immutability for a new object storage repository. In governance mode, a storage administrator with the appropriate permissions can override the immutability settings and delete the protected backups from the repository.

    For more information about the immutability of primary object storage repositories, refer to the user guide:

    https://helpcenter.veeam.com/docs/vbo365/guide/immutability.html?ver=80

    https://helpcenter.veeam.com/docs/vbo365/guide/adding_object_storage.html?ver=80

    S3 Storage budget with performance

    You can now meet your restore time objectives and storage budget using the AWS S3 One Zone-Infrequent Access (IA) storage class. This storage class is ideal for data accessed infrequently but requires quick retrieval. S3 One Zone-IA provides a cost-effective solution by storing data in a single availability zone.

    AWS S3 One Zone-IA stores data in a single AZ, which is a good choice for storing secondary backup copies at competitive costs. You can also use it as cost-effective storage for data replicated from another AWS Region using S3 Cross-Region Replication.

    But this release supports this storage class as a backup and backup copy job target.

    For more information:   

    https://helpcenter.veeam.com/docs/vbo365/guide/amazon_storage_class.html?zoom_highlight=s3+one+zone-infrequent&ver=80

    Scalability and performance: Proxy Pools, Linux, and NATS server

    In version 8, Veeam Backup for Microsoft 365 introduces a backup proxy pool for large-scale environments with multiple backup proxy servers. This logical grouping allows several backup proxy servers to function as a single entity, enabling simultaneous processing of Microsoft 365 data during backup, backup copy, and restore operations. It enhances scalability and load balancing.

    You can also deploy Linux backup proxies as standalone units or as part of a pool to simplify backup infrastructure management and lower total cost of ownership (TCO). It is achieved using Linux hosts, with Red Hat and Canonical Ubuntu support.

    There is another new component: the NATS server. The NATS server ensures communication between backup proxy servers so that you can group them into a backup proxy pool for more information about backup proxy pools.

    The NATS server utilizes JetStream technology for message queuing and streaming. Veeam Backup for Microsoft 365 leverages this technology to establish streams facilitating communication between backup infrastructure components.

    Centralized PostgreSQL database and cache

    A PostgreSQL instance hosts a centralized configuration database for Veeam Backup for Microsoft 365 components, including backup proxy servers. PostgreSQL is now required to manage the product databases, including the configuration database, organization cache databases, and repository persistent caches.

    It’s important to note that, in previous versions, the repository’s persistent cache was stored on the proxy server. In the current version, it is now located in the PostgreSQL instance.

    Cache reduces costs associated with cloud storage providers by minimizing the need to write to or read from the object storage repository. For instance, when using Veeam Explorers to access backups in the object storage repository, Veeam Backup for Microsoft 365 retrieves the backup structure from the cache.

    This structure is then displayed in the inventory pane of each Explorer, allowing you to navigate it without downloading data from the object storage repository.

    You can use your existing PostgreSQL server or install a new instance as part of the Veeam Backup for Microsoft 365 deployment.

    Management and security enhancements

    You can now manage a list of users and user groups with permissions to access the Veeam Backup for Microsoft 365 console and Veeam Explorers. Assigning the Veeam Backup Administrator role to these users and user groups grants them full administrative rights within Veeam Backup for Microsoft 365.

    Veeam Backup for Microsoft 365 v8 supports multi-factor authentication (MFA) for additional user verification.

    Now, improving security by using a dedicated custom user account to run a backup proxy service for Windows and Linux proxies is also possible.

    For more information:

    https://helpcenter.veeam.com/docs/vbo365/guide/manage_users_roles.html?ver=80

    https://helpcenter.veeam.com/docs/vbo365/guide/mfa.html?ver=80

    https://helpcenter.veeam.com/docs/vbo365/guide/add_linux_proxy_service_acc.html?ver=80

    Complete list of new features and enhancements

    You can access the complete list here (there are many):

    I hope this post was helpful!

  • Veeam Data Platform release 12.2: New Features and Integrations

    Veeam Data Platform release 12.2: New Features and Integrations

    Again, Veeam has exceeded expectations and delivered a new release for the Veeam Data Platform with many new features and integrations.

    This release expands Veeam Backup and Replication platform capabilities significantly. This post will focus on Proxmox VE compatibility, MongoDB support, Nutanix Prism Central integration, and Microsoft Azure Cosmos DB support.

    The VDP 12.2 for VBR release also delivers other new workload integrations, enabling customers to transition to new platforms with confidence and security, like:

    • IBM Db2: Linux on Power support and SLES 15 SP5 support.
    • SAP HANA plugin: now operates on RHEL 9.4 for SAP HANA.
    • Veeam Agent for Linux: support for the latest Linux distribution versions like SLES 15 SP6 (for x86_64), Debian 12.4, and Fedora 40.
    • VMware Cloud Director 10.6 support and vSphere 8.0 Update 3 support.
    • Amazon Redshift Cluster and Amazon FSx protection.
    • Microsoft Azure Data Lake Storage Gen2 protection.

    The new release brings many other new features and improvements.

    I can highlight the latest “Direct to Archive” Scale-out Backup Repository capability. Now, it is possible to offload aging backups directly from the Performance Tier to the Archive Tier without requiring offloaded backups to land on the Capacity Tier first.

    I would also like to emphasize the addition of immutable snapshots for HPE storage arrays. HPE 3PAR, Primera, Alletra, and Alletra MP storage users now have an enhanced immutability option. With version 12.2, HPE’s native Virtual Lock functionality can make storage snapshots immutable, providing an extra layer of security against data tampering and ransomware by creating read-only snapshots.

    Another key highlight is optimizing API integration with Object Storage repositories (SOSAPI?). The new version significantly reduces the load on object storage by cutting the frequency of specific API calls by up to 10 times. This decrease in API calls can help lower costs with cloud object storage providers who charge based on API usage.

    Proxmox Virtual Environment

    Native support for host-based backups of Proxmox VE virtual machines (VMs) offers users enhanced flexibility to meet evolving virtualization and cloud requirements.

    Key highlights of Veeam’s Proxmox VE integration include:

    Immutable backup support: ensures that both on-premises and cloud backups of Proxmox VE VMs remain secure from ransomware encryption, accidental deletions, or malicious tampering during cyberattacks through storage-based immutability.

    Enhanced backup performance: with advanced changed block tracking (CBT) integration and HotAdd backup mode, Proxmox VE users can benefit from rapid and efficient VM backups (including powered-off VM), reducing the impact of lengthy backups and ensuring smooth daily operations.

    Veeam BitLooker: Veeam BitLooker: This feature improves data security in backups by integrating with Microsoft BitLocker encryption. It optimizes backup performance and conserves storage space by automatically excluding unused disk blocks, including those from deleted guest OS files.

    Backup storage flexibility: integration with Proxmox support for all Veeam Backup & Replication backup repository types, including hardened repositories and object storage! It´s great!

    Freedom of full VM recovery and portability between hypervisors: the virtual machines from most significant virtualization platformsin the market (VMware vSphere, Microsoft Hyper-V, Nutanix AHV, Red Hat Virtualization, Oracle Linux Virtualization Manager) or public clouds (AWS, Azure, Google Cloud) can now be quickly restored to Proxmox VE (and vice versa), minimizing downtime and boosting productivity. Additionally, physical server backups can be restored to Proxmox VMs, streamlining disaster recovery and P2V migrations.

    Advanced granular recovery capabilities: itincludes swiftly recovering all modified and deleted files, such as those lost due to a ransomware attack, ensuring business continuity, and providing peace of mind during unexpected data loss or disruptions.

    Requirements for Proxmox VE Infrastructure:

    • Virtualization Platform: compatible with Proxmox VE version 8.2 or later. It supports both standalone and clustered Proxmox VE configurations. Enterprise and No-Subscription repositories are supported, though support for the No-Subscription repository is considered experimental.
    • Proxmox VE Virtual Machines: all types and versions of virtual hardware are supported, except Open vSwitch networking. All operating systems compatible with the Proxmox VE version are supported; file-level restore is also supported for many file systems.

    For more information about Veeam Backup for Proxmox VE, please refer to the user guide: https://helpcenter.veeam.com/docs/vbproxmoxve/userguide/overview.html?ver=1

    MongoDB Backup

    Native support for backing up and restoring MongoDB on Linux, the world’s most popular NoSQL database, allows you to perform backups at the replica set level without needing pre-freeze and post-thaw scripts.

    Veeam designed MongoDB Backup to offer a seamless, straightforward, and intuitive user experience within Veeam Backup & Replication while preserving the familiar MongoDB OpsManager interface.

    It ensures that DBAs and backup administrators can efficiently work with our solution, even without in-depth knowledge of Veeam Backup & Replication or MongoDB.

    In summary, MongoDB Enterprise is a self-managed version of MongoDB that organizations deploy on their infrastructure, whether on-premises, in data centers, or on cloud servers. It includes advanced features like LDAP integration, Kerberos authentication, encryption, and auditing.

    MongoDB Enterprise also supports additional tools like MongoDB Ops Manager for managing deployments. This version can be deployed on any infrastructure, offering flexibility in hosting the database on physical servers or virtual machines.

    The key highlights of Veeam’s MongoDB Backup integration include:

    Simple discovery: add your MongoDB replica sets to the new, dedicated type of protection group, and Veeam will automatically identify all replica set members, discover the application topology, and install the necessary backup components on each replica set node.

    Backup policy flexibility: allows you to configure policies with either individual replica sets or entire protection groups. It also features a unique option to choose the preferred node for data retrieval, either manually or automatically. It ensures that protected data is retrieved only once, reducing the impact of backup activities on the production environment.

    Backup storage flexibility: support all Veeam Backup & Replication backup repository types, including hardened and object storage!

    Use of a proven backup engine: MongoDB protection utilizes the Veeam Agent for Linux volume-level backup engine. This integration provides immediate access to all rich features included in this agent.

    Veeam Explorer for MongoDB: Yes, there is a Veeam Explorer for MongoDB, offering the flexibility to restore individual collections and databases for routine operational restores or entire MongoDB instances for disaster recovery to either their original or alternative locations

    Requirements for MongoDB:

    • MongoDB 7.0, MongoDB 6.0, or MongoDB 5.0
    • Only the following 64-bit versions of the operating systems are supported for the database server: Debian 9 or later, RHEL 6.2 or later, CentOS 6.2 or later, Oracle Linux 6.2 or later, Rocky 8.0 or later, Alma 8.0 or later, SLES 12 SP4 – SP5, SLES 15 SP1 – SP5, Ubuntu 16.04 or later.

    When you rescan the MongoDB replica set (a group of instances that maintain the same data set) for the first time, Veeam Backup & Replication deploys necessary components on each computer where the MongoDB primary daemon process (“mongod”) is detected.

    To protect MongoDB databases with Veeam Backup & Replication, you must perform the following operations:

    • Discover computers hosting MongoDB data and deploy Veeam components on them.
    • Configure an application backup policy to guide Veeam components in creating MongoDB backups.
    • Execute and manage MongoDB backup operations.
    • In case of malware activity or accidental actions, restore MongoDB items from backups stored in Veeam repositories. You can restore entire instances or specific collections from a MongoDB backup, with support for restoring to the original location or an alternate one.

    For more information about Veeam Backup for Proxmox VE, please refer to the user guide:

    https://helpcenter.veeam.com/docs/backup/plugins/mongo_backup.html?ver=120

    Nutanix Prism Central and AHV integration improvements

    Prism Central offers a centralized workspace for monitoring and managing multiple clusters. It provides a single pane of glass for overseeing many Nutanix clusters, simplifying administration by consolidating management tasks into one interface.

    Prism Central offers detailed insights and analytics into the performance and health of Nutanix clusters. It includes real-time monitoring, performance metrics, and alerts, helping administrators proactively address issues before they impact operations. It enables simplified tasks such as software upgrades, capacity planning, and resource optimization across multiple clusters.

    Prism Central includes automation features that streamline routine operations, such as automated capacity management and workload balancing. The resource optimization capabilities help optimize resource utilization and ensure efficient use of infrastructure across clusters.

    In this way, administrators can manage and monitor their entire Nutanix infrastructure across different clouds or data centers from this unified dashboard, and this Veeam integration is welcome in the IT and cloud operational teams.

    Prism Central Integration: streamline your backup operations across clusters managed by Prism Central with a single backup appliance, creating a unified and user-friendly backup architecture.

    Dynamic Backup Job Scope: utilize Prism Central VM categories (or “tags”) to automatically include VMs in backup jobs based on their category affiliation.

    REST API Enhancements: improved REST API functionality now includes expanded Prism Central management capabilities and enhanced network management features.

    Requirements for Prism Central and AOS (Acropolis OS):

    • Veeam Backup for Nutanix AHV is compatible with Nutanix AOS versions 6.5.x – 6.7.x and Prism Central version pc.2023.6 (or later).

    The Prism Central deployment scenario enables you to safeguard workloads across multiple clusters registered with Nutanix Prism Central. This setup offers a centralized web console for managing backup and restore operations for all registered clusters, streamlining the process and minimizing the time needed to install, configure, and maintain Veeam Backup for Nutanix AHV.

    The Prism Central deployment scenario is similar to the standalone cluster deployment scenario that is also available (both can coexist). In this case, the Veeam Backup for Nutanix AHV architecture comprises the following set of components:

    • Nutanix AHV Prism Central.
    • Nutanix AHV clusters.
    • Backup server.
    • Backup appliance.
    • Nutanix AHV Plug-in.
    • Backup repositories.
    • Workers.

    For more information about the Prism Deployment deployment scenario, please refer to the user guide:

    https://helpcenter.veeam.com/docs/vbahv/userguide/infrastructure_prism_central.html?ver=6

    Other expanded capabilities for Nutanix AHV available in the release 12.2 are:

    Backup from secondary VM copy: in Nutanix environments with replication enabled, you can enable backups from the VM replica with a single checkbox. This feature offloads backup operations from production clusters, minimizing the impact on production environments.

    Multiple network adapter support: AHV appliances and workers can utilize multiple network interfaces to enhance data processing efficiency.

    Advanced job settings: you can now adjust the job compression level and block size and configure BitLooker options to meet your needs.

    Gmail and Microsoft 365 support email notifications

    New resources view: The updated VM inventory view in the web interface offers a comprehensive summary of protection status across managed clusters and allows for adding VMs to existing protection policies with just one click.

    Microsoft Azure Cosmos DB protection

    Azure Cosmos DB is a fully managed, globally distributed NoSQL and relational database service provided by Microsoft Azure. It is designed to offer high availability, low latency, and scalable performance for applications requiring a global reach.

    The service is designed to allow customers to elastically (and independently) scale throughput and storage across any number of geographical regions. Azure Cosmos DB is a globally distributed, multi-model database that is used in a wide range of applications and use cases.

    Azure Cosmos DB transparently replicates the data to all the regions associated with your Azure Cosmos DB account.

    Veeam Backup for Microsoft Azure manages backup policies to create backups of Cosmos DB accounts. A backup policy consists of various settings that dictate how backups are executed, including which data to back up, where to store the backups, and when to initiate the backup process.

    Veeam Backup for Microsoft Azure leverages native Microsoft Azure features rather than installing agent software to back up Cosmos DB account data. During each backup session, it captures a point-in-time snapshot (timestamp) for each Cosmos DB account included in the backup policy.

    Veeam Backup for Microsoft Azure also allows you to back up Cosmos DB for PostgreSQL clusters.

    For more information about the backup and restore to the Cosmos DB Accounts, please refer to the user guide:

    https://helpcenter.veeam.com/docs/vbazure/guide/overview_cosmos_db.html?ver=70

    References:

    https://veeam.com/backup-replication-download.html

    https://www.veeam.com/kb4600

    https://www.veeam.com/company/press-release/new-veeam-data-platform-v12-2-extends-1-data-resilience-to-more-platforms-and-applications.html