大型门户网站建设功能,小程序建站网站,百度推广登录首页,怎么搭建一个博客网站综述
首先#xff0c;本篇文章所介绍的内容#xff0c;已经有完整的实现#xff0c;可以参考这里。 在微服务、DevOps和云平台流行的当下#xff0c;使用一个高效的持续集成工具也是一个非常重要的事情。虽然市面上目前已经存在了比较成熟的自动化构建工具#xff0c;比如…综述
首先本篇文章所介绍的内容已经有完整的实现可以参考这里。 在微服务、DevOps和云平台流行的当下使用一个高效的持续集成工具也是一个非常重要的事情。虽然市面上目前已经存在了比较成熟的自动化构建工具比如jekines还有一些商业公司推出的自动化构建工具但他们都不能够很好的和云环境相结合。那么究竟该如何实现一个简单、快速的基于云环境的自动化构建系统呢我们首先以一个Springboot应用为例来介绍一下整体的发布流程然后再来看看具体如何实现。发布的步骤大体如下 1.首先从代码仓库下载代码比如Gitlab、GitHub等 2.接着是进行打包比如使用Maven、Gradle等 3.如果要使用k8s作为编排还需要把步骤2产生的包制作成镜像比如用Docker等 4.上传步骤3的镜像到远程仓库比如Harhor、DockerHub等 5.最后下载镜像并编写Deployment文件部署到k8s集群 如图1所示 图1
从以上步骤可以看出发布过程中需要的工具和环境至少包括代码仓库Gitlab、GitHub等、打包环境Maven、Gradle等、镜像制作Docker等、镜像仓库Harbor、DockerHub等、k8s集群等此外还包括发布系统自身的数据存储等。 可以看出整个流程里依赖的环境很多如果发布系统不能与这些环境解耦那么要想实现一个安装简单、功能快速的系统没有那么容易。那么有没有合理的解决方案来实现与这些环境的解耦呢答案是有的下面就分别介绍。
代码仓库
操作代码仓库一般系统提供的都有对应Restful API以GitLab系统提供的Java客户端为例如下代码
dependencygroupIdorg.gitlab4j/groupIdartifactIdgitlab4j-api/artifactIdversion4.17.0/version
/dependency比如我们想获取某个项目的分支列表如下代码所示
public ListBranch branchList(CodeRepo codeRepo, BranchListParam param) {GitLabApi gitLabApi gitLabApi(codeRepo);ListBranch list null;try {list gitLabApi.getRepositoryApi().getBranches(param.getProjectIdOrPath(), param.getBranchName());} catch (GitLabApiException e) {LogUtils.throwException(logger, e, MessageCodeEnum.PROJECT_BRANCH_PAGE_FAILURE);} finally {gitLabApi.close();}
}private GitLabApi gitLabApi(CodeRepo codeRepo) {GitLabApi gitLabApi new GitLabApi(codeRepo.getUrl(), codeRepo.getAuthToken());gitLabApi.setRequestTimeout(1000, 5 * 1000);try {gitLabApi.getVersion();}catch(GitLabApiException e) {//如果token无效则用账号登录if(e.getHttpStatus() 401 !StringUtils.isBlank(codeRepo.getAuthUser())) {gitLabApi new GitLabApi(codeRepo.getUrl(), codeRepo.getAuthUser(), codeRepo.getAuthPassword());gitLabApi.setRequestTimeout(1000, 5 * 1000);}}return gitLabApi;
}打包环境
我们以Maven为例进行说明一般情况下我们使用Maven打包时需要首先安装Maven环境接着引入打包插件然后使用mvn clean package命令就可以打包了。比如springboot自带插件
plugingroupIdorg.springframework.boot/groupIdartifactIdspring-boot-maven-plugin/artifactIdversion2.5.6/versionconfigurationclassifierexecute/classifiermainClasscom.test.Application/mainClass/configurationexecutionsexecutiongoalsgoalrepackage/goal/goals/execution/executions
/plugin再比如通用的打包插件
plugingroupIdorg.apache.maven.plugins/groupIdartifactIdmaven-assembly-plugin/artifactIdversion3.8.2/versionconfigurationappendAssemblyIdfalse/appendAssemblyIddescriptorsdescriptorsrc/main/resources/assemble.xml/descriptor/descriptorsoutputDirectory../target/outputDirectory/configurationexecutionsexecutionidmake-assembly/idphasepackage/phasegoalsgoalsingle/goal/goals/execution/executions
/plugin等等。然后再通过运行mvn clean package命令进行打包。那么在打包时如果要去除对maven环境的依赖该如何实现呢 可以使用嵌入式maven插件maven-embedder来实现。 具体可以这样来做首先在平台项目里引入依赖如下
dependencygroupIdorg.apache.maven/groupIdartifactIdmaven-embedder/artifactIdversion3.8.1/version
/dependency
dependencygroupIdorg.apache.maven/groupIdartifactIdmaven-compat/artifactIdversion3.8.1/version
/dependency
dependencygroupIdorg.apache.maven.resolver/groupIdartifactIdmaven-resolver-connector-basic/artifactIdversion1.7.1/version
/dependency
dependencygroupIdorg.apache.maven.resolver/groupIdartifactIdmaven-resolver-transport-http/artifactIdversion1.7.1/version
/dependency运行如下代码就可以对项目进行打包了
String[] commands new String[] { clean, package, -Dmaven.test.skip };
String pomPath D:/hello/pom.xml;
MavenCli cli new MavenCli();
try {cli.doMain(commands, pomPath, System.out, System.out);
} catch (Exception e) {e.printStackTrace();
}但是一般情况下我们通过maven的settings文件还会做一些配置比如配置工作目录、nexus私服地址、Jdk版本、编码方式等等如下
?xml version1.0 encodingUTF-8?
settings xmlnshttp://maven.apache.org/SETTINGS/1.0.0xmlns:xsihttp://www.w3.org/2001/XMLSchema-instancexsi:schemaLocationhttp://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsdlocalRepositoryC:/m2/repository/localRepositoryprofilesprofileidmyNexus/idrepositoriesrepositoryidnexus/idnamenexus/nameurlhttps://repo.maven.apache.org/maven2/urlreleasesenabledtrue/enabled/releasessnapshotsenabledtrue/enabled/snapshots/repository/repositoriespluginRepositoriespluginRepositoryidnexus/idnamenexus/nameurlhttps://repo.maven.apache.org/maven2/urlreleasesenabledtrue/enabled/releasessnapshotsenabledtrue/enabled/snapshots/pluginRepository/pluginRepositories/profileprofileidjava11/idactivationactiveByDefaulttrue/activeByDefaultjdk11/jdk/activationpropertiesmaven.compiler.source11/maven.compiler.sourcemaven.compiler.target11/maven.compiler.targetmaven.compiler.compilerVersion11/maven.compiler.compilerVersionproject.build.sourceEncodingUTF-8/project.build.sourceEncodingproject.build.outputEncodingUTF-8/project.build.outputEncoding/properties/profile/profilesactiveProfilesactiveProfilemyNexus/activeProfile/activeProfiles
/settings通过查看MavenCli类发现doMain(CliRequest cliRequest)方法有比较丰富的参数CliRequest的代码如下
package org.apache.maven.cli;public class CliRequest
{String[] args;CommandLine commandLine;ClassWorld classWorld;String workingDirectory;File multiModuleProjectDirectory;boolean debug;boolean quiet;boolean showErrors true;Properties userProperties new Properties();Properties systemProperties new Properties();MavenExecutionRequest request;CliRequest( String[] args, ClassWorld classWorld ){this.args args;this.classWorld classWorld;this.request new DefaultMavenExecutionRequest();}public String[] getArgs(){return args;}public CommandLine getCommandLine(){return commandLine;}public ClassWorld getClassWorld(){return classWorld;}public String getWorkingDirectory(){return workingDirectory;}public File getMultiModuleProjectDirectory(){return multiModuleProjectDirectory;}public boolean isDebug(){return debug;}public boolean isQuiet(){return quiet;}public boolean isShowErrors(){return showErrors;}public Properties getUserProperties(){return userProperties;}public Properties getSystemProperties(){return systemProperties;}public MavenExecutionRequest getRequest(){return request;}public void setUserProperties( Properties properties ) {this.userProperties.putAll( properties ); }
}可以看出这些参数非常丰富也许可以满足我们的需求但是CliRequest只有一个默认修饰符的构造方法也就说只有位于org.apache.maven.cli包下的类才有访问CliRequest构造方法的权限我们可以在平台项目里新建一个包org.apache.maven.cli然后再创建一个类如DefaultCliRequest继承自CliRequest然后实现一个public的构造方法就可以在任何包里使用该类了如下代码
package org.apache.maven.cli;import org.codehaus.plexus.classworlds.ClassWorld;public class DefaultCliRequest extends CliRequest{public DefaultCliRequest(String[] args, ClassWorld classWorld) {super(args, classWorld);}public void setWorkingDirectory(String directory) {this.workingDirectory directory;}
}定义好参数类型DefaultCliRequest后我们再来看看打包的代码
public void doPackage() {String[] commands new String[] { clean, package, -Dmaven.test.skip };DefaultCliRequest request new DefaultCliRequest(commands, null);request.setWorkingDirectory(D:/hello/pom.xml);Repository repository new Repository();repository.setId(nexus);repository.setName(nexus);repository.setUrl(https://repo.maven.apache.org/maven2);RepositoryPolicy policy new RepositoryPolicy();policy.setEnabled(true);policy.setUpdatePolicy(always);policy.setChecksumPolicy(fail);repository.setReleases(policy);repository.setSnapshots(policy);String javaVesion 11;Profile profile new Profile();profile.setId(java11);Activation activation new Activation();activation.setActiveByDefault(true);activation.setJdk(javaVesion);profile.setActivation(activation);profile.setRepositories(Arrays.asList(repository));profile.setPluginRepositories(Arrays.asList(repository));Properties properties new Properties();properties.put(java.home, D:/java/jdk-11.0.16.2);properties.put(java.version, javaVesion);properties.put(maven.compiler.source, javaVesion);properties.put(maven.compiler.target, javaVesion);properties.put(maven.compiler.compilerVersion, javaVesion);properties.put(project.build.sourceEncoding, UTF-8);properties.put(project.reporting.outputEncoding, UTF-8);profile.setProperties(properties);MavenExecutionRequest executionRequest request.getRequest();executionRequest.setProfiles(Arrays.asList(profile));MavenCli cli new MavenCli();try {cli.doMain(request);} catch (Exception e) {e.printStackTrace();}
}如果需要设置其他参数也可以通过以上参数自行添加。
镜像制作
一般情况下我们在Docker环境中通过Docker命令来制作镜像过程如下 1.首先编写Dockerfile文件 2.通过docker build制作镜像 3.通过docker push上传镜像 可以看出如果要使用docker制作镜像的话必须要有docker环境而且需要编写Dockerfile文件。当然也可以不用安装docker环境直接使用doker的远程接口post/build。但是在远程服务器中仍然需要安装doker环境和编写Dockerfile。在不依赖Docker环境的情况下仍然可以制作镜像下面就介绍一款工具Jib的用法。 Jib是谷歌开源的一套工具github地址它是一个无需Docker守护进程——也无需深入掌握Docker最佳实践的情况下为Java应用程序构建Docker和OCI镜像 它可以作为Maven和Gradle的插件也可以作为Java库。
比如使用jib-maven-plugin插件构建镜像的代码如下
plugingroupIdcom.google.cloud.tools/groupIdartifactIdjib-maven-plugin/artifactIdversion3.3.0/versionconfigurationfromimageopenjdk:13-jdk-alpine/image/fromtoimagegcr.io/dhorse/client/imagetagstag102/tag/tagsauth!--连接镜像仓库的账号和密码 --usernameusername/usernamepasswordpassword/password/auth/tocontainerportsport8080/port/ports/container/configurationexecutionsexecutionphasepackage/phasegoalsgoalbuild/goal/goals/execution/executions
/plugin然后使用命令进行构建
mvn compile jib:build可以看出无需docker环境就可以实现镜像的构建。但是要想通过平台类型的系统去为每个系统构建镜像显然通过插件的方式不太合适因为需要每个被构建系统引入jib-maven-plugin插件才行也就是需要改造每一个系统这样就会带来一定的麻烦。那么有没有不需要改造系统的方式直接进行构建镜像呢答案是通过Jib-core就可以实现。
首先在使用Jib-core的项目中引入依赖maven如下
dependencygroupIdcom.google.cloud.tools/groupIdartifactIdjib-core/artifactIdversion0.22.0/version
/dependency然后就可以直接使用Jib-core的API来进行制作镜像如下代码
try {JibContainerBuilder jibContainerBuilder null;if (StringUtils.isBlank(context.getProject().getBaseImage())) {jibContainerBuilder Jib.fromScratch();} else {jibContainerBuilder Jib.from(context.getProject().getBaseImage());}//连接镜像仓库5秒超时System.setProperty(jib.httpTimeout, 5000);System.setProperty(sendCredentialsOverHttp, true);String fileNameWithExtension targetFiles.get(0).toFile().getName();ListString entrypoint Arrays.asList(java, -jar, fileNameWithExtension);RegistryImage registryImage RegistryImage.named(context.getFullNameOfImage()).addCredential(context.getGlobalConfigAgg().getImageRepo().getAuthUser(),context.getGlobalConfigAgg().getImageRepo().getAuthPassword());jibContainerBuilder.addLayer(targetFiles, /).setEntrypoint(entrypoint).addVolume(AbsoluteUnixPath.fromPath(Paths.get(/etc/localtime))).containerize(Containerizer.to(registryImage).setAllowInsecureRegistries(true).addEventHandler(LogEvent.class, logEvent - logger.info(logEvent.getMessage())));
} catch (Exception e) {logger.error(Failed to build image, e);return false;
}其中targetFiles是要构建镜像的目标文件比如springboot打包后的jar文件。 通过Jib-core可以很轻松的实现镜像构建而不需要依赖任何其他环境也不需要被构建系统做任何改造非常方便。
镜像仓库
类似代码仓库提供的Restful API也可以通过Restful API来操作镜像仓库以Harbor创建一个项目为例代码如下
public void createProject(ImageRepo imageRepo) {String uri api/v2.0/projects;if(!imageRepo.getUrl().endsWith(/)) {uri / uri;}HttpPost httpPost new HttpPost(imageRepo.getUrl() uri);RequestConfig requestConfig RequestConfig.custom().setConnectionRequestTimeout(5000).setConnectTimeout(5000).setSocketTimeout(5000).build();httpPost.setConfig(requestConfig);httpPost.setHeader(Content-Type, application/json;charsetUTF-8);httpPost.setHeader(Authorization, Basic Base64.getUrlEncoder().encodeToString((imageRepo.getAuthUser() : imageRepo.getAuthPassword()).getBytes()));ObjectNode objectNode JsonUtils.getObjectMapper().createObjectNode();objectNode.put(project_name, dhorse);//1公有类型objectNode.put(public, 1);httpPost.setEntity(new StringEntity(objectNode.toString(),UTF-8));try (CloseableHttpResponse response createHttpClient(imageRepo.getUrl()).execute(httpPost)){if (response.getStatusLine().getStatusCode() ! 201 response.getStatusLine().getStatusCode() ! 409) {LogUtils.throwException(logger, response.getStatusLine().getReasonPhrase(),MessageCodeEnum.IMAGE_REPO_PROJECT_FAILURE);}} catch (IOException e) {LogUtils.throwException(logger, e, MessageCodeEnum.IMAGE_REPO_PROJECT_FAILURE);}
}k8s集群
同样k8s也提供了Restful API。同时官方也提供了各种语言的客户端下面以Java语言的客户端为例来创建一个deployment。 首先引入Maven依赖
dependencygroupIdio.kubernetes/groupIdartifactIdclient-java/artifactIdversion13.0.0/version
/dependency然后使用如下代码
public boolean createDeployment(DeployContext context) {V1Deployment deployment new V1Deployment();deployment.apiVersion(apps/v1);deployment.setKind(Deployment);deployment.setMetadata(deploymentMetaData(context.getDeploymentAppName()));deployment.setSpec(deploymentSpec(context));ApiClient apiClient this.apiClient(context.getCluster().getClusterUrl(),context.getCluster().getAuthToken(), 1000, 1000);AppsV1Api api new AppsV1Api(apiClient);CoreV1Api coreApi new CoreV1Api(apiClient);String namespace context.getProjectEnv().getNamespaceName();String labelSelector K8sUtils.getDeploymentLabelSelector(context.getDeploymentAppName());try {V1DeploymentList oldDeployment api.listNamespacedDeployment(namespace, null, null, null, null,labelSelector, null, null, null, null, null);if (CollectionUtils.isEmpty(oldDeployment.getItems())) {deployment api.createNamespacedDeployment(namespace, deployment, null, null, null);} else {deployment api.replaceNamespacedDeployment(context.getDeploymentAppName(), namespace, deployment, null, null,null);}} catch (ApiException e) {if (!StringUtils.isBlank(e.getMessage())) {logger.error(Failed to create k8s deployment, message: {}, e.getMessage());} else {logger.error(Failed to create k8s deployment, message: {}, e.getResponseBody());}return false;}return true;
}private ApiClient apiClient(String basePath, String accessToken, int connectTimeout, int readTimeout) {ApiClient apiClient new ClientBuilder().setBasePath(basePath).setVerifyingSsl(false).setAuthentication(new AccessTokenAuthentication(accessToken)).build();apiClient.setConnectTimeout(connectTimeout);apiClient.setReadTimeout(readTimeout);return apiClient;
}至此关键的技术点已经介绍完了更多内容请参考这里
