.NET应用系统的国际化-基于Roslyn抽取词条、更新代码

  • .NET应用系统的国际化-基于Roslyn抽取词条、更新代码已关闭评论
  • 243 次浏览
  • A+
所属分类:.NET技术
摘要

上篇文章我们介绍了VUE+.NET应用系统的国际化-多语言词条服务系统国际化改造整体设计思路如下:

上篇文章我们介绍了

VUE+.NET应用系统的国际化-多语言词条服务

系统国际化改造整体设计思路如下:

  1. 提供一个工具,识别前后端代码中的中文,形成多语言词条,按语言、界面、模块统一管理多有的多语言词条
  2. 提供一个翻译服务,批量翻译多语言词条
  3. 提供一个词条服务,支持后端代码在运行时根据用户登录的语言,动态获取对应的多语言文本
  4. 提供前端多语言JS生成服务,按界面动态生成对应的多语言JS文件,方便前端VUE文件使用。
  5. 提供代码替换工具,将VUE前端代码中的中文替换为$t("词条ID"),后端代码中的中文替换为TermService.Current.GetText("词条ID")

今天,我们在上篇文章的基础上,继续介绍基于Roslyn抽取词条、更新代码。

一、业务背景

先说一下业务背景,后端.NET代码中存在大量的中文提示和异常消息,甚至一些中文返回值文本。

这些中文文字都需要识别出来,抽取为多语言词条,同时将代码替换为调用多语言词条服务获取翻译后的文本。

例如:

private static void CheckMd5(string fileName, string md5Data) {       string md5Str = MD5Service.GetMD5(fileName);       if (!string.Equals(md5Str, md5Data, StringComparison.OrdinalIgnoreCase))       {            throw new CustomException(PackageExceptionConst.FileMd5CheckFailed, "服务包文件MD5校验失败:" + fileName);       } }

代码中需要将“服务包文件MD5校验失败”这个文本做多语言改造。

这里通过调用多语言词条服务I18NTermService,根据线程上下文中设置的语言,获取对应的翻译文本。例如以下代码:

var text=T.Core.I18N.Service.TermService.Current.GetTextFormatted("词条ID""默认文本"); 

throw new CustomException(PackageExceptionConst.FileMd5CheckFailed, text + fileName);

以上背景下,我们准备使用Roslyn技术对代码进行中文扫描,对扫描出来的文本,做词条抽取、代码替换。

二、使用Roslyn技术对代码进行中文扫描

首先,我们先定义好代码中多语言词条的扫描结果类TermScanResult

 1  [Serializable]  2     public class TermScanResult  3     {  4         public Guid Id { get; set; }  5         public string OriginalText { get; set; }  6   7         public string ChineseText { get; set; }  8   9         public string SlnName { get; set; } 10  11         public string ProjectName { get; set; } 12  13         public string ClassFile { get; set; } 14  15         public string MethodName { get; set; } 16  17         public string Code { get; set; } 18  19         public I18NTerm I18NTerm { get; set; } 20  21         public string SlnPath { get; set; } 22  23         public string ClassPath { get; set; } 24 28         public string SubSystemCode { get; set; } 29  30         public override string ToString() 31         { 32             return Code; 33         } 34     }

上述代码中SubSystemCode是一个业务管理维度。大家忽略即可。

我们会以sln解决方案为单位,扫描代码中的中文文字。

以下是具体的实现代码

public async Task<List<TermScanResult>> CheckSln(string slnPath, System.ComponentModel.BackgroundWorker backgroundWorker, SubSystemFile subSystemFiles, string subSystem) {             var slnFile = new FileInfo(slnPath);             var results = new List<TermScanResult>();              MSBuildHelper.RegisterMSBuilder();             var solution = await MSBuildWorkspace.Create().OpenSolutionAsync(slnPath);              var subSystemInfo = subSystemFiles?.SubSystemSlnMappings.FirstOrDefault(w => w.SlnName.Select(s => s += ".sln").Contains(slnFile.Name.ToLower()));              if (solution.Projects != null && solution.Projects.Count() > 0)             {                 foreach (var project in solution.Projects.ToList())                 {                     backgroundWorker.ReportProgress(10, $"扫描Project: {project.Name}");                     var documents = project.Documents.Where(x => x.Name.Contains(".cs"));                      if (project.Name.ToLower().Contains("test"))                     {                         continue;                     }                     var codeReplace = new CodeReplace();                     foreach (var document in documents)                     {                         var tree = await document.GetSyntaxTreeAsync();                         var root = tree.GetCompilationUnitRoot();                         if (root.Members == null || root.Members.Count == 0) continue;                         //member                         var classDeclartions = root.DescendantNodes().Where(i => i is ClassDeclarationSyntax);                          foreach (var classDeclare in classDeclartions)                         {                             var programDeclaration = classDeclare as ClassDeclarationSyntax;                             if (programDeclaration == null) continue;                              foreach (var memberDeclarationSyntax in programDeclaration.Members)                             {                                 foreach (var item in GetLiteralStringExpression(memberDeclarationSyntax))                                 {                                     var statementCode = item.Item1;                                     foreach (var syntaxNode in item.Item3)                                     {                                         ExpressionSyntaxParser expressionSyntaxParser = new ExpressionSyntaxParser();                                         var text = "";                                         var expressionSyntax = expressionSyntaxParser                                             .GetExpressionSyntaxVerifyRule(syntaxNode as ExpressionSyntax, statementCode);                                         if (expressionSyntax != null)                                         {                                             // 排除                                             if (expressionSyntaxParser.IsExcludeCaller(expressionSyntax, statementCode))                                             {                                                 continue;                                             }                                              text = expressionSyntaxParser.GetExpressionSyntaxOriginalText(expressionSyntax, statementCode);                                             if (expressionSyntax is Microsoft.CodeAnalysis.CSharp.Syntax.InterpolatedStringExpressionSyntax)                                             {                                                 text = expressionSyntaxParser.GetExpressionSyntaxOriginalText(expressionSyntax, statementCode);                                                  if (expressionSyntax is Microsoft.CodeAnalysis.CSharp.Syntax.LiteralExpressionSyntax)                                                 {                                                     if (!expressionSyntax.IsKind(SyntaxKind.StringLiteralExpression))                                                     {                                                         continue;                                                     }                                                     text = expressionSyntax.NormalizeWhitespace().ToString();                                                 }                                             }                                         }                                         if (CheckChinese(text) == false) continue;                                         if (string.IsNullOrWhiteSpace(text)) continue;                                         if (string.IsNullOrWhiteSpace(text.Replace(""", "").Trim())) continue;                                          results.Add(new TermScanResult()                                         {                                             Id = Guid.NewGuid(),                                             ClassPath = programDeclaration.SyntaxTree.FilePath,                                             SlnPath = slnPath,                                             OriginalText = text.Replace(""", "").Trim(),                                             ChineseText = text,                                             SlnName = slnFile.Name,                                             ProjectName = project.Name,                                             ClassFile = programDeclaration.Identifier.Text,                                             MethodName = item.Item2,                                             Code = statementCode,                                             SubSystemCode = subSystem                                         });                                     }                                 }                             }                         }                     }                 }             }       return results; }

上述代码中,我们先使用MSBuilder编译,构建 sln解决方案

MSBuildHelper.RegisterMSBuilder(); var solution = await MSBuildWorkspace.Create().OpenSolutionAsync(slnPath);

然后遍历solution下的各个Project中的class类

foreach (var project in solution.Projects.ToList())
var documents = project.Documents.Where(x => x.Name.Contains(".cs"));

然后遍历类中声明、成员、方法中的每行代码,通过正则表达式识别是否有中文字符

public static bool CheckChinese(string strZh) {             Regex re = new Regex(@"[u4e00-u9fa5]+");             if (re.IsMatch(strZh))             {                 return true;             }             return false; }

如果存在中文字符,作为扫描后的结果,识别为多语言词条

results.Add(new TermScanResult() {         Id = Guid.NewGuid(),         ClassPath = programDeclaration.SyntaxTree.FilePath,         SlnPath = slnPath,         OriginalText = text.Replace(""", "").Trim(),         ChineseText = text,         SlnName = slnFile.Name,         ProjectName = project.Name,         ClassFile = programDeclaration.Identifier.Text,         MethodName = item.Item2,         Code = statementCode,        //管理维度                                           SubSystemCode = subSystem    //管理维度
});

TermScanResult中没有对词条属性赋值。

public I18NTerm I18NTerm { get; set; }

下一篇文章的代码中,我们会通过多语言翻译服务,将翻译后的文本放到I18NTerm 属性中,作为多语言词条。

三、代码替换

代码替换这块逻辑中,我们设计了一个类SourceWeaver,对上一步的代码扫描结果,进行代码替换

CodeScanReplace这个方法中完成了代码的二次扫描和替换
 /// <summary>     /// 源代码替换服务     /// </summary>     public class SourceWeaver     {         List<CommonTermDto> commonTerms = new List<CommonTermDto>();         List<CommonTermDto> commSubTerms = new List<CommonTermDto>();          public SourceWeaver()         {             commonTerms = JsonConvert.DeserializeObject<List<CommonTermDto>>(File.ReadAllText("comm_data.json"));             commSubTerms = JsonConvert.DeserializeObject<List<CommonTermDto>>(File.ReadAllText("comm_sub_data.json"));         }         public async Task CodeScanReplace(Tuple<List<I18NTerm>, List<TermScanResult>> result, System.ComponentModel.BackgroundWorker backgroundWorker)         {             try             {                 backgroundWorker.ReportProgress(0, "正在对代码进行替换.");                 var termScanResultGroupBy = result.Item2.GroupBy(g => g.SlnName);                 foreach (var termScanResult in termScanResultGroupBy)                 {                     var termScan = termScanResult.FirstOrDefault();                     MSBuildHelper.RegisterMSBuilder();                     var solution = await MSBuildWorkspace.Create().OpenSolutionAsync(termScan.SlnPath).ConfigureAwait(false);                     if (solution.Projects.Any())                     {                         foreach (var project in solution.Projects.ToList())                         {                             if (project.Name.ToLower().Contains("test"))                             {                                 continue;                             }                             var projectTermScanResults = result.Item2.Where(f => f.ProjectName == project.Name);                              var documents = project.Documents.Where(x =>                             {                                 return x.Name.Contains(".cs") && projectTermScanResults.Any(f => $"{f.ClassPath}" == x.FilePath);                             });                              foreach (var document in documents)                             {                                 var tree = await document.GetSyntaxTreeAsync().ConfigureAwait(false);                                 var root = tree.GetCompilationUnitRoot();                                 if (root.Members.Count == 0) continue;                                  var classDeclartions = root.DescendantNodes()                                     .Where(i => i is ClassDeclarationSyntax);                                 List<MemberDeclarationSyntax> syntaxNodes = new List<MemberDeclarationSyntax>();                                 foreach (var classDeclare in classDeclartions)                                 {                                     if (!(classDeclare is ClassDeclarationSyntax programDeclaration)) continue;                                     var className = programDeclaration.Identifier.Text;                                      foreach (var method in programDeclaration.Members)                                     {                                         if (method is ConstructorDeclarationSyntax)                                         {                                             syntaxNodes.Add((ConstructorDeclarationSyntax)method);                                         }                                         else if (method is MethodDeclarationSyntax)                                         {                                             syntaxNodes.Add((MethodDeclarationSyntax)method);                                         }                                         else if (method is PropertyDeclarationSyntax)                                         {                                             syntaxNodes.Add(method);                                         }                                         else if (method is FieldDeclarationSyntax)                                         {                                             // 注:常量不支持                                             syntaxNodes.Add(method);                                         }                                     }                                 }                                  var terms = termScanResult.Where(                                     f => f.ProjectName == document.Project.Name && f.ClassPath == document.FilePath).ToList();                                 backgroundWorker.ReportProgress(10, $"正在检查{document.FilePath}文件.");                                 ReplaceNodesAndSave(root, syntaxNodes, terms, result, backgroundWorker, document.Name);                             }                         }                     }                 }             }             catch (Exception ex)             {                 LogUtils.LogError(string.Format("异常类型:{0}rn异常消息:{1}rn异常信息:{2}rn",                     ex.GetType().Name, ex.Message, ex.StackTrace));                 backgroundWorker.ReportProgress(0, ex.Message);             }         }          public async void ReplaceNodesAndSave(SyntaxNode classSyntaxNode, List<MemberDeclarationSyntax> syntaxNodes, IEnumerable<TermScanResult> terms, Tuple<List<I18NTerm>, List<TermScanResult>> result,             System.ComponentModel.BackgroundWorker backgroundWorker, string className)         {              {//check pro是否存在词条                 if (AppConfig.Instance.IsCheckTermPro)                 {                     backgroundWorker.ReportProgress(15, $"词条验证中.");                     var termsCodes = terms.Select(f => f.I18NTerm.Code).ToList();                     var size = 100;                     var p = (result.Item2.Count() + size - 1) / size;                      using DBHelper dBHelper = new DBHelper();                     List<I18NTerm> items = new List<I18NTerm>();                     for (int i = 0; i < p; i++)                     {                         var list = termsCodes                             .Skip(i * size).Take(size);                         Thread.Sleep(10);                         var segmentItems = await dBHelper.GetTermsAsync(termsCodes).ConfigureAwait(false);                         items.AddRange(segmentItems);                     }                      List<TermScanResult> termScans = new List<TermScanResult>();                     foreach (var term in terms)                     {                         if (items.Any(f => f.Code == term.I18NTerm.Code))                         {                             termScans.Add(term);                         }                         else                         {                             backgroundWorker.ReportProgress(20, $"词条{term.OriginalText}未导入到词条库,该词条将忽略替换.");                         }                     }                     terms = termScans;                 }             }              var newclassDeclare = classSyntaxNode;             newclassDeclare = classSyntaxNode.ReplaceNodes(syntaxNodes,                     (methodDeclaration, _) =>                     {                                              MemberDeclarationSyntax newMemberDeclarationSyntax = methodDeclaration;                         var className = ((ClassDeclarationSyntax)newMemberDeclarationSyntax.Parent).Identifier.Text;                         List<StatementSyntax> statementSyntaxes = new List<StatementSyntax>();                          switch (newMemberDeclarationSyntax)                         {                             case ConstructorDeclarationSyntax:                                 {                                     var blockSyntax = (newMemberDeclarationSyntax as ConstructorDeclarationSyntax).NormalizeWhitespace().Body;                                     if (blockSyntax == null)                                     {                                         break;                                     }                                     foreach (var statement in blockSyntax.Statements)                                     {                                         var nodeStatement = statement.DescendantNodes();                                          statementSyntaxes.Add(new CodeReplace().ReplaceStatementNodes(statement,                                             new ExpressionSyntaxParser().LiteralStringExpression(nodeStatement), terms, commonTerms, commSubTerms));                                     }                                      break;                                 }                              case MethodDeclarationSyntax:                                 {                                     var blockSyntax = (methodDeclaration as MethodDeclarationSyntax).NormalizeWhitespace().Body;                                     if (blockSyntax == null)                                     {                                         break;                                     }                                     foreach (var statement in blockSyntax.Statements)                                     {                                         var nodeStatement = statement.DescendantNodes();                                         statementSyntaxes.Add(new CodeReplace().ReplaceStatementNodes(statement,                                                new ExpressionSyntaxParser().LiteralStringExpression(nodeStatement), terms, commonTerms, commSubTerms));                                     }                                      break;                                 }                              case PropertyDeclarationSyntax:                                 {                                     var propertyDeclarationSyntax = newMemberDeclarationSyntax as PropertyDeclarationSyntax;                                      var nodeStatement = propertyDeclarationSyntax.DescendantNodes();                                      return new CodeReplace().ReplacePropertyNodes(newMemberDeclarationSyntax as PropertyDeclarationSyntax,                                         new ExpressionSyntaxParser().LiteralStringExpression(nodeStatement), terms, commonTerms, commSubTerms);                                 }                              case FieldDeclarationSyntax:                                 {                                     var fieldDeclarationSyntax = newMemberDeclarationSyntax as FieldDeclarationSyntax;                                     var nodeStatement = fieldDeclarationSyntax.DescendantNodes();                                     return new CodeReplace().ReplaceFiledNodes(fieldDeclarationSyntax,                                            new ExpressionSyntaxParser().LiteralStringExpression(nodeStatement), terms, commonTerms, commSubTerms);                                 }                         }                         backgroundWorker.ReportProgress(50, $"解析并对类文件{className}中的方法做语句替换.");                         // 替换方法内部                         if (newMemberDeclarationSyntax is MethodDeclarationSyntax)                         {                             return new CodeReplace().ReplaceMethodDeclaration(newMemberDeclarationSyntax as MethodDeclarationSyntax, statementSyntaxes);                         }                         else if (newMemberDeclarationSyntax is ConstructorDeclarationSyntax)                         {                             return new CodeReplace().ReplaceConstructorDeclaration(newMemberDeclarationSyntax as ConstructorDeclarationSyntax, statementSyntaxes);                         }                         return newMemberDeclarationSyntax;                     });              var sourceStr = newclassDeclare.NormalizeWhitespace().GetText().ToString();             File.WriteAllText(newclassDeclare.SyntaxTree.FilePath, sourceStr);             backgroundWorker.ReportProgress(100, $"完成{className}的替换.");         }     }

关键的代码语义替换的实现代码:

 public StatementSyntax ReplaceStatementNodes(StatementSyntax statement, List<ExpressionSyntax> expressionSyntaxes, IEnumerable<TermScanResult> terms             , List<CommonTermDto> commonTerms, List<CommonTermDto> commSubTerms)         {             var statementSyntax = statement.ReplaceNodes(expressionSyntaxes, (syntaxNode, _) =>             {                 var statementStr = statement.NormalizeWhitespace().ToString();                  var argumentLists = statement.DescendantNodes().                                                OfType<InvocationExpressionSyntax>();                 ExpressionSyntaxParser expressionSyntaxParser = new ExpressionSyntaxParser();                 return expressionSyntaxParser.ExpressionSyntaxTermReplace(syntaxNode, statementStr, terms, commonTerms, commSubTerms);              });              return statementSyntax;         }

这里,我们抽象了一个ExpressionSyntaxParser 类,负责替换代码:

T.Core.I18N.Service.TermService.Current.GetTextFormatted
 public ExpressionSyntax ExpressionSyntaxTermReplace(ExpressionSyntax syntaxNode, string statementStr, IEnumerable<TermScanResult> terms             , List<CommonTermDto> commonTerms, List<CommonTermDto> commSubTerms)         {             var expressionSyntax = GetExpressionSyntaxVerifyRule(syntaxNode, statementStr);             var originalText = GetExpressionSyntaxOriginalText(expressionSyntax, statementStr);              var I18Expr = "";             var interpolationSyntaxes = syntaxNode.DescendantNodes().OfType<InterpolationSyntax>();                      var term = terms.FirstOrDefault(i => i.ChineseText == originalText);              if (term == null)                 return syntaxNode;             string termcode = term.I18NTerm.Code; if (syntaxNode is InterpolatedStringExpressionSyntax)             {                 if (interpolationSyntaxes.Count() > 0)                 {                     var parms = "";                     foreach (var item in interpolationSyntaxes)                     {                         parms += $",{item.ToString().TrimStart('{').TrimEnd('}')}";                     }                     I18Expr = "$"{T.Core.I18N.Service.TermService.Current.GetTextFormatted("" + termcode + "", " + originalText + parms + ")}"";                     var token1 = SyntaxFactory.Token(default, SyntaxKind.StringLiteralToken, I18Expr, "", default);                     return SyntaxFactory.LiteralExpression(SyntaxKind.StringLiteralExpression, token1);                 }                 else                 {                      var startToken = SyntaxFactory.Token(SyntaxKind.InterpolatedStringStartToken);                     if ((syntaxNode as InterpolatedStringExpressionSyntax).StringStartToken.Value == startToken.Value)                     {                         // 如果本身有"$"                         I18Expr = "$"{T.Core.I18N.Service.TermService.Current.GetText("" + termcode + ""," + originalText + ")}";                     }                     else                     {                         // 如果没有"$"                         I18Expr = "$"{T.Core.I18N.Service.TermService.Current.GetText("" + termcode + "",\teld"" + originalText + "")}";                         I18Expr = I18Expr.Replace("\teld", "$");                     }                 }             }             else             {                 I18Expr = "$"{T.Core.I18N.Service.TermService.Current.GetText("" + termcode + ""," + originalText + ")}";             }              var token = SyntaxFactory.Token(default(SyntaxTriviaList), SyntaxKind.InterpolatedVerbatimStringStartToken, I18Expr, "$"", default(SyntaxTriviaList));             var literalExpressionSyntax = SyntaxFactory.InterpolatedStringExpression(token);             return literalExpressionSyntax;         }

T.Core.I18N.Service.TermService这个就是多语言词条服务类,这个类中提供了一个GetText的方法,通过词条编号,获取多语言文本。

代码完成替换后,打开VS,对工程引用多语言词条服务的Nuget包/dll,重新编译代码,手工校对替换后的代码即可。
以上是.NET应用系统的国际化-基于Roslyn抽取词条、更新代码的分享。



周国庆
2023/3/19