How to migrate revisions that doesn't exist in the target?

[marsijanin]

Hi!

We got two on premises Azure DevOps servers. The target server is the copy of the source sever before update 2022 Update 2 to AzureDevOps 25H2. Because of this issue all attaches become unavaliable. After the update there was some changes, so we have restored from backup before the update a new server instances (the target) and trying to migrate changes, than has ben made after the update

After manual updating of the ReflectedWorkItemId into a work item on the targer server I'm truing to migrate new revisions from the source with the following config

{
  "$schema": "https://devopsmigration.io/schema/configuration.schema.json",
  "Serilog": {
    "MinimumLevel": "Information"
  },
  "MigrationTools": {
    "Version": "16.3",
    "Endpoints": {
      "Source": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://BADSERVER/tfs/SrvNccCollection/",
        "Project": "PRG_Prime",
        "Authentication": {
			"AuthenticationMode": "Prompt"
        },
        "LanguageMaps": {
          "AreaPath": "Area",
          "IterationPath": "Iteration"
        },
        "ProductVersion": "OnPremises",
		"Enabled": true
      },
      "Target": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://NEWSERVER/tfs/SrvNccCollection/",
        "Project": "PRG_Prime",
        "Authentication": {
          "AuthenticationMode": "Prompt"
        },
        "ReflectedWorkItemIdField": "ReflectedWorkItemId",
        "LanguageMaps": {
          "AreaPath": "Area",
          "IterationPath": "Iteration"
        },
        "ProductVersion": "OnPremises",
		"Enabled": true
      }
    },
    "Processors": [
      {
        "ProcessorType": "TfsWorkItemMigrationProcessor",
        "Enabled": true,
	    "PauseAfterEachWorkItem": false,
        "WIQLQuery": "SELECT [System.Id] FROM WorkItems WHERE [System.TeamProject] = @TeamProject AND [System.Id] IN (84632)",
        "FixHtmlAttachmentLinks": true,
        "WorkItemCreateRetryLimit": 5,
        "FilterWorkItemsThatAlreadyExistInTarget": false,
        "GenerateMigrationComment": true,
        "SourceName": "Source",
        "TargetName": "Target",
		"AttachRevisionHistory": true
      }
    ],
    "CommonTools": {
	"TfsNodeStructureTool": {
        "Areas": {
          "Mappings": [
            {
              "Match": "^NSMS",
              "Replacement": "PRG_Prime"
            },
			{
              "Match": "^Rollout",
              "Replacement": "PRG_Prime"
            },
			{
              "Match": "^NVS",
              "Replacement": "PRG_Prime"
            },
			{
              "Match": "^Prg_PrimeC",
              "Replacement": "PRG_Prime"
            }
          ]
        },
        "Enabled": true,
        "Iterations": {
          "Mappings": [
            {
              "Match": "^NSMS",
              "Replacement": "PRG_Prime"
            },
			{
              "Match": "^Rollout",
              "Replacement": "PRG_Prime"
            },
			{
              "Match": "^NVS",
              "Replacement": "PRG_Prime"
            },
			{
              "Match": "^Prg_PrimeC",
              "Replacement": "PRG_Prime"
            }
          ]
        },
		
        "ReplicateAllExistingNodes": true,
        "ShouldCreateMissingRevisionPaths": true
      },
	  "TfsRevisionManagerTool": {
                "Enabled":  true,
                "ReplayRevisions": true,
				"MaxRevisions": 10000
            },
		"WorkItemTypeMappingTool": { 
		"Enabled":true,
		"Mappings": { "Bug": "Bug" }
		},
		"TfsWorkItemTypeValidatorTool": {
			"ExcludeWorkItemTypes": [
				"Change Request", "Code Review Request", "Code Review Response", "Epic", "Feature", "Feedback Request", "Feedback Response", "Issue", "Release", "Request", "Requirement", "Review", "Risk", "Roadmap", "SDL Bug", "Shared Parameter", "Shared Steps", "Smart Goal", "Task", "Test Case", "Test Plan", "Test Suite"
			],
			"ExcludeSourceFields": {
				"Bug": [
					"Infotecs.ALM.DocumentationChange",
					"Infotecs.ALM.BugLanguage",
					"Infotecs.ALM.Branch",
					"Infotecs.ALM.Components",
					"Infotecs.ALM.Modules",
					"Microsoft.VSTS.CMMI.RootCause",
					"Microsoft.VSTS.CMMI.Blocked",
					"Microsoft.VSTS.Common.Severity",
					"Microsoft.VSTS.Common.Discipline",
					"Microsoft.VSTS.Build.IntegrationBuild",
					"Microsoft.VSTS.Build.FoundIn",
					"Microsoft.VSTS.Common.Triage",
					"Microsoft.VSTS.Common.Priority",
					"Microsoft.VSTS.Common.ResolvedReason",
					"System.Reason",
					"System.State"
				]
			}
		},
      "FieldMappingTool": {
        "Enabled": true,
        "FieldMaps": [
          {
            "FieldMapType": "FieldLiteralMap",
            "targetField": "Custom.SomeField",
            "value": "New field value",
            "ApplyTo": [
              "SomeWorkItemType"
            ]
          }
        ]
      }
    }
  }
}

According to the log the utility sees the revisions diffrence, but finaly says Found 0 revisions to migrate on Work item:Source and leaves the target work item unupdated:

[10:22:02 INF] [16.3.2] [                 Bug][Complete:     1/1][sid: 84632|Rev: 53][tid:  null | Work Item has 53 revisions and revision migration is set to True
[10:22:02 INF] [16.3.2] Source: Found 53 revisions to migrate on  Work item:Source
[10:22:02 INF] [16.3.2] Target: Found 52 revisions to migrate on  Work item:Target
[10:22:02 INF] [16.3.2] Source: Found 0 revisions to migrate on  Work item:Source
[10:22:02 INF] [16.3.2] [                 Bug][Complete:     1/1][sid: 84632|Rev: 53][tid:  null | Attachemnts 1 | LinkMigrator:True
[10:22:02 INF] [16.3.2] AttachmentMigrationEnricher: Migrating  1 attachemnts from 84632 to 84632
[10:22:02 ERR] [16.3.2] Exception downloading attachements
System.Exception: HTTP Status Code : InternalServerError - TF51334: An unknown Web service error occurred: Specified cast is not valid.. Check the Event Log for more information.
   at Microsoft.TeamFoundation.WorkItemTracking.Proxy.WorkItemServer.DownloadFile(Int32 fileAttachmentId)
   at MigrationTools.Tools.TfsAttachmentTool.ExportAttachment(Attachment wia, String exportpath) in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.TfsObjectModel\Tools\TfsAttachmentTool.cs:line 131
Error logged to Elmah.io!
!! Check for latest version - We fix issues constantly - If not, please create a discussion on https://github.com/nkdAgility/azure-devops-migration-tools/discussions so we can get this fixed !!
[10:22:23 INF] [16.3.2] [                 Bug][Complete:     1/1][sid: 84632|Rev: 53][tid:  null | Links 2 | LinkMigrator:True
[10:22:23 INF] [16.3.2] [SKIP] Source and Target have same number of links  84632 - Bug
[10:22:23 INF] [16.3.2] GitRepositoryEnricher: Enriching 84632 To fix Git Repo Links
[10:22:23 INF] [16.3.2] EmbededImagesRepairEnricher: Fixing HTML field attachments for work item 84632 from https://msk-tfs-t.infotecs-nt/tfs/SrvNccCollection/ to https://msk-tfs-tn.infotecs-nt/tfs/SrvNccCollection/
[10:22:23 INF] [16.3.2] TfsWorkItemEmbededLinkTool: Fixing embedded mention links on target work item 84632 from https://msk-tfs-t.infotecs-nt/tfs/SrvNccCollection/ to https://msk-tfs-tn.infotecs-nt/tfs/SrvNccCollection/
[10:22:23 INF] [16.3.2] [                 Bug][Complete:     1/1][sid: 84632|Rev: 53][tid:  null | Skipping as work item exists and no revisions to sync detected
[10:22:23 INF] [16.3.2] [                 Bug][Complete:     1/1][sid: 84632|Rev: 53][tid:  null | Average time of 22.333 per work item and 0 hours 0 minutes 0 seconds estimated to completion.
[10:22:23 INF] [16.3.2]  Migration Processor Complete TfsWorkItemMigrationProcessor
[10:22:23 INF] [16.3.2] TfsWorkItemMigrationProcessor completed in 00:00:00
[10:22:23 INF] [16.3.2] Command ExecuteMigrationCommand completed in 00:00:46.0746719

We have tried many variants of the TfsRevisionManagerTool confg section .We also have tried to disable attaches migration (the erros in the log's listing is because of the specified issue) whit the same results.

We have also tryed to use Tfs WorkItem Bulk Edit Processor, but we have failed to fount working configuration this that processor.

Could You, please, provide a working example of configuration.json for our migration scenario, or such scenario (migrate only revisions, than are newer) is unsupported?

[MrHinsh]

The system can only migreate revissions from a source to a target when the source revision is newer than the target. Can you check if the target has been "edited" more recently than the source for that work item?

[marsijanin]

The problem was into revision's dates. I updated the target workitem (ReflectedWorkItemId) and it become newer. Manualy uptaing the source workitem solve the porblem.

Another question - how to disable attachments migration? Is seting Enabled to false into the TfsAttachmentTool enough?

[marsijanin]

We have tried this config

{
  "$schema": "https://devopsmigration.io/schema/configuration.schema.json",
  "Serilog": {
    "MinimumLevel": "Information"
  },
  "MigrationTools": {
    "Version": "16.3",
    "Endpoints": {
      "Source": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://SOURCE/tfs/SrvNccCollection/",
        "Project": "PRG_Prime",
        "Authentication": {
			"AuthenticationMode": "Prompt"
        },
        "LanguageMaps": {
          "AreaPath": "Area",
          "IterationPath": "Iteration"
        },
        "ProductVersion": "OnPremises",
		"Enabled": true
      },
      "Target": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://TARGET/tfs/SrvNccCollection/",
        "Project": "PRG_Prime",
        "Authentication": {
          "AuthenticationMode": "Prompt"
        },
        "ReflectedWorkItemIdField": "ReflectedWorkItemId",
        "LanguageMaps": {
          "AreaPath": "Area",
          "IterationPath": "Iteration"
        },
        "ProductVersion": "OnPremises",
		"Enabled": true
      }
    },
    "Processors": [
      {
        "ProcessorType": "TfsWorkItemMigrationProcessor",
        "Enabled": true,
	    "PauseAfterEachWorkItem": false,
        "WIQLQuery": "SELECT [System.Id] FROM WorkItems WHERE [System.TeamProject] = @TeamProject AND [System.Id] IN (117862)",
        "FixHtmlAttachmentLinks": true,
        "WorkItemCreateRetryLimit": 5,
        "FilterWorkItemsThatAlreadyExistInTarget": false,
        "GenerateMigrationComment": true,
        "SourceName": "Source",
        "TargetName": "Target",
		"AttachRevisionHistory": false
      }
    ],
    "CommonTools": {
	"TfsAttachmentTool": {
	   "Enabled": false
	},
	"TfsNodeStructureTool": {
        "Areas": {
          "Mappings": [
            {
              "Match": "^NSMS",
              "Replacement": "PRG_Prime"
            },
			{
              "Match": "^Rollout",
              "Replacement": "PRG_Prime"
            },
			{
              "Match": "^NVS",
              "Replacement": "PRG_Prime"
            },
			{
              "Match": "^Prg_PrimeC",
              "Replacement": "PRG_Prime"
            }
          ]
        },
        "Enabled": true,
        "Iterations": {
          "Mappings": [
            {
              "Match": "^NSMS",
              "Replacement": "PRG_Prime"
            },
			{
              "Match": "^Rollout",
              "Replacement": "PRG_Prime"
            },
			{
              "Match": "^NVS",
              "Replacement": "PRG_Prime"
            },
			{
              "Match": "^Prg_PrimeC",
              "Replacement": "PRG_Prime"
            }
          ]
        },
		
        "ReplicateAllExistingNodes": true,
        "ShouldCreateMissingRevisionPaths": true
      },
	  "TfsRevisionManagerTool": {
                "Enabled":  true,
                "ReplayRevisions": true,
				"MaxRevisions": 10000
            },
		"WorkItemTypeMappingTool": { 
		"Enabled":true,
		"Mappings": { "Bug": "Bug" }
		},
		"TfsWorkItemTypeValidatorTool": {
			"ExcludeWorkItemTypes": [
				"Change Request", "Code Review Request", "Code Review Response", "Epic", "Feature", "Feedback Request", "Feedback Response", "Issue", "Release", "Request", "Requirement", "Review", "Risk", "Roadmap", "SDL Bug", "Shared Parameter", "Shared Steps", "Smart Goal", "Task", "Test Case", "Test Plan", "Test Suite"
			],
			
      "FieldMappingTool": {
        "Enabled": true,
        "FieldMaps": [
          {
            "FieldMapType": "FieldLiteralMap",
            "targetField": "Custom.SomeField",
            "value": "New field value",
            "ApplyTo": [
              "SomeWorkItemType"
            ]
          }
        ]
      }
    }
  }
}

[15:16:30 INF] [16.3.2] EmbededImagesRepairEnricher: Fixing HTML field attachments for work item 117862 from https://SOURCE/tfs/SrvNccCollection/ to https://msk-tfs-tn.infotecs-nt/tfs/SrvNccCollection/
[15:16:30 WRN] [16.3.2] EmbededImagesRepairEnricher: Failed to download image https://SOURCE/tfs/SrvNccCollection/ae6570fa-5368-4f02-a362-759a8119c561/_apis/wit/attachments/4fd2b109-fbc8-4a2d-b3c0-5fd57a1b38a8?fileName=image.png from WorkItem 117862, Field Repro Steps. Status: 500 (Internal Server Error)
[15:16:30 ERR] [16.3.2] EmbededImagesRepairEnricher: Unable to fix HTML field attachments for work item 117862 from https://SOURCE/tfs/SrvNccCollection/ to https://msk-tfs-tn.infotecs-nt/tfs/SrvNccCollection/
System.Net.Http.HttpRequestException: Response status code does not indicate success: 500 (Internal Server Error).
   at MigrationTools.Tools.TfsEmbededImagesTool.UploadedAndRetrieveAttachmentLinkUrl(String matchedSourceUri, String sourceFieldName, WorkItemData targetWorkItem, String sourcePersonalAccessToken) in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.TfsObjectModel\Tools\TfsEmbededImagesTool.cs:line 177
   at MigrationTools.Tools.TfsEmbededImagesTool.FixEmbededImages(WorkItemData wi, String oldTfsurl, String newTfsurl, String sourcePersonalAccessToken) in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.TfsObjectModel\Tools\TfsEmbededImagesTool.cs:line 102
Error logged to Elmah.io!
!! Check for latest version - We fix issues constantly - If not, please create a discussion on https://github.com/nkdAgility/azure-devops-migration-tools/discussions so we can get this fixed !!
[15:16:51 INF] [16.3.2] TfsWorkItemEmbededLinkTool: Fixing embedded mention links on target work item 117862 from https://SOURCE/tfs/SrvNccCollection/ to https://msk-tfs-tn.infotecs-nt/tfs/SrvNccCollection/
[15:16:52 INF] [16.3.2] [                 Bug][Complete:     1/1][sid:117862|Rev: 27][tid:  null |  Saved TargetWorkItem 117862. Replayed revision 27 of 1
[15:16:52 INF] [16.3.2] [                 Bug][Complete:     1/1][sid:117862|Rev: 27][tid:  null | Links 1 | LinkMigrator:True
[15:16:52 INF] [16.3.2] [SKIP] Source and Target have same number of links  117862 - Bug
[15:16:52 INF] [16.3.2] GitRepositoryEnricher: Enriching 117862 To fix Git Repo Links
[15:16:52 INF] [16.3.2] [                 Bug][Complete:     1/1][sid:117862|Rev: 27][tid:  null | ...Saved as 117862
[15:16:52 INF] [16.3.2] [                 Bug][Complete:     1/1][sid:117862|Rev: 27][tid:  null | Average time of 23.114 per work item and 0 hours 0 minutes 0 seconds estimated to completion.
[15:16:52 INF] [16.3.2]  Migration Processor Complete TfsWorkItemMigrationProcessor
[15:16:52 INF] [16.3.2] TfsWorkItemMigrationProcessor completed in 00:00:00
[15:16:52 INF] [16.3.2] Command ExecuteMigrationCommand completed in 00:00:47.3679571
[15:16:52 INF] [16.3.2] Check the logs for errors: c:\Distr\MigrationTools-16.3.2\logs\20260414151604

How to disable image migration? Setting "FixHtmlAttachmentLinks": to false got the same effect - images become unavaliable after the migration

[marsijanin]

We found another way using "FieldMapType": "RegexFieldMap"