update and deploy modules in an isolated network environment

Poster Content
nk4um User
Posts: 27
June 29, 2015 13:35

Hi Peter,

Thank you very much for your efforts. Now it works gracefully :D

Regards, Yang

Posted by pjr (View)
Hi Yang - looks like you're on the right track. You have found a bug in clean-up after the NK instance has been installed to the maven repository. Everything had worked but the temp freeze directory was being left behind and an error thrown.

We just fixed this and I've successfully tested a full round trip freeze/thaw of an NK instance with this build.gradle...

apply plugin: 'netkernel'

apply plugin: 'maven'

apply plugin: 'maven-publish'

buildscript {
	repositories {
	   mavenLocal()
	   mavenCentral()
	   maven {
		   url "http://maven.netkernel.org/netkernel-maven"
	   }
	}

	dependencies {
	   classpath group: 'urn.org.netkernel', name: 'gradle.plugin', version: '[1.1.8,)'
	}
}

//For Freezing
uploadArchives { repositories { mavenDeployer { repository(url: "file:/home/pjr/.m2/repository/") } } }

//For Thawing
repositories {
    maven {
        url "file:/home/pjr/.m2/repository/" 
    }
}

netkernel {

   instances {
		//Test for freezeInstallXXXX Task
       PROD
       {
           edition = "SE"
           location = "/home/pjr/NKSE-5.2.1"
           freeze group: "be.milieuinfo.lod", name: "netkernel-base-production", version : "1.0.0"
       }

	//Test for thawXXXXX Task
	PROD2
       {
           edition = "SE"
           location = "/home/pjr/NKSE-5.2.1-thawed"
           thaw group: "be.milieuinfo.lod", name: "netkernel-base-production", version : "1.0.0"
       }
   }

}

Please can you try this (note its now using 1.1.8 as the minimum gradle plugin version).

Cheers,

Peter

Like · Post Reply
nk4um Moderator
Posts: 901
June 29, 2015 13:24

Hi Yang - looks like you're on the right track. You have found a bug in clean-up after the NK instance has been installed to the maven repository. Everything had worked but the temp freeze directory was being left behind and an error thrown.

We just fixed this and I've successfully tested a full round trip freeze/thaw of an NK instance with this build.gradle...

apply plugin: 'netkernel'

apply plugin: 'maven'

apply plugin: 'maven-publish'

buildscript {
	repositories {
	   mavenLocal()
	   mavenCentral()
	   maven {
		   url "http://maven.netkernel.org/netkernel-maven"
	   }
	}

	dependencies {
	   classpath group: 'urn.org.netkernel', name: 'gradle.plugin', version: '[1.1.8,)'
	}
}

//For Freezing
uploadArchives { repositories { mavenDeployer { repository(url: "file:/home/pjr/.m2/repository/") } } }

//For Thawing
repositories {
    maven {
        url "file:/home/pjr/.m2/repository/" 
    }
}

netkernel {

   instances {
		//Test for freezeInstallXXXX Task
       PROD
       {
           edition = "SE"
           location = "/home/pjr/NKSE-5.2.1"
           freeze group: "be.milieuinfo.lod", name: "netkernel-base-production", version : "1.0.0"
       }

	//Test for thawXXXXX Task
	PROD2
       {
           edition = "SE"
           location = "/home/pjr/NKSE-5.2.1-thawed"
           thaw group: "be.milieuinfo.lod", name: "netkernel-base-production", version : "1.0.0"
       }
   }

}

Please can you try this (note its now using 1.1.8 as the minimum gradle plugin version).

Cheers,

Peter

Like · Post Reply
1 person likes this post
nk4um User
Posts: 27
June 26, 2015 12:33

Hi Peter,

OK, I realized that I don't need to build it from source. It's already in official Netkernel repository. My fault.

So I have already created a complete netkernel instance locally, and I am going to create an artifact with installFreeze.

My build.gradle file looks like

([ View])

apply plugin: 'netkernel'

apply plugin: 'maven'

apply plugin: 'maven-publish'

netkernel plugin buildscript {

   repositories {
       mavenLocal()
       mavenCentral()
       maven {
           url "http://maven.netkernel.org/netkernel-maven"
       }
   }
   dependencies {
       classpath group: 'urn.org.netkernel', name: 'gradle.plugin', version: '[1.1.7,)'
   }

}

netkernel {

uploadArchives { repositories { mavenDeployer { repository(url: "file:/home/yyz/.m2/repository/") } } }

   instances {
       PROD
       {
           edition = "SE"
           location = "/home/yyz/Downloads/netkernel-production/"
           freeze group: "be.milieuinfo.lod", name: "netkernel-base-production", version : "1.0.0"
       }
   }

}

However the execution result is:

([ View])

yyz@ubuntu:~/netkernel-build$ gradle installFreezePROD

WARNING: Could not find module.xml in the project - only sysadmin tasks will be available

copyBeforeFreezePROD
freezeTidyPROD
freezePROD
installFreezePROD FAILED

FAILURE: Build failed with an exception.

  • What went wrong:

Execution failed for task ':installFreezePROD'. > Could not find method delete() for arguments /home/yyz/Downloads/netkernel-production-freeze/netkernel-base-production.jar on task ':installFreezePROD'.

  • Try:

Run with --stacktrace option to get the stack trace. Run with --info or --debu option to get more log output.

BUILD FAILED

Total time: 49.325 secs

What could be the reason? Maybe I miss something again.

Thank you and have a nice day.

Regards, Yang

Like · Post Reply
nk4um User
Posts: 27
June 26, 2015 09:01

Hi Peter,

Thanks a lot for your suggestion. Unfortunately I am confused about using the plugin.

So this is a gradle plugin, I didn't find published artifact so I guess I have to build it from source code. Then I git clone that repository, and perform a "gradle build" at the the root directory of the plugin. Am I correct?

However the build is failed, with the following information:

([ View])

compileJava UP-TO-DATE
compileGroovy UP-TO-DATE
processResources UP-TO-DATE
classes UP-TO-DATE
jar UP-TO-DATE
assemble UP-TO-DATE
compileTestJava UP-TO-DATE
compileTestGroovy

startup failed: Could not instantiate global transform class org.spockframework.compiler.SpockTransform specified at jar:file:/home/yyz/.gradle/caches/modules-2/files-2.1/org.spockframework/spock-core/0.7-groovy-1.8/3a677d19e8d3acf3bd296c4023356256d55da5a3/spock-core-0.7-groovy-1.8.jar!/META-INF/services/org.codehaus.groovy.transform.ASTTransformation because of exception org.spockframework.util.IncompatibleGroovyVersionException: The Spock compiler plugin cannot execute because Spock 0.7.0-groovy-1.8 is not compatible with Groovy 2.3.10. For more information, see http://versioninfo.spockframework.org Spock location: file:/home/yyz/.gradle/caches/modules-2/files-2.1/org.spockframework/spock-core/0.7-groovy-1.8/3a677d19e8d3acf3bd296c4023356256d55da5a3/spock-core-0.7-groovy-1.8.jar Groovy location: file:/usr/lib/gradle/2.4/lib/groovy-all-2.3.10.jar

1 error

compileTestGroovy FAILED

FAILURE: Build failed with an exception.

  • What went wrong:

Execution failed for task ':compileTestGroovy'. > Compilation failed; see the compiler error output for details.

  • Try:

Run with --info or --debug option to get more log output.

  • Exception is:

org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':compileTestGroovy'. at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:69) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46) at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35) at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64) at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58) at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42) at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52) at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53) at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43) at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:310) at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:79) at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:63) at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:51) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:23) at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:88) at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:37) at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62) at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23) at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:68) at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32) at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62) at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:55) at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:149) at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106) at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86) at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:90) at org.gradle.tooling.internal.provider.ExecuteBuildActionRunner.run(ExecuteBuildActionRunner.java:28) at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35) at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:41) at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:28) at org.gradle.launcher.exec.DaemonUsageSuggestingBuildActionExecuter.execute(DaemonUsageSuggestingBuildActionExecuter.java:50) at org.gradle.launcher.exec.DaemonUsageSuggestingBuildActionExecuter.execute(DaemonUsageSuggestingBuildActionExecuter.java:27) at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:40) at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:169) at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237) at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210) at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35) at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24) at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206) at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169) at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33) at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22) at org.gradle.launcher.Main.doAction(Main.java:33) at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45) at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54) at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35) at org.gradle.launcher.GradleMain.main(GradleMain.java:23) Caused by: org.gradle.api.internal.tasks.compile.CompilationFailedException: Compilation failed; see the compiler error output for details. at org.gradle.api.internal.tasks.compile.ApiGroovyCompiler.execute(ApiGroovyCompiler.java:159) at org.gradle.api.internal.tasks.compile.ApiGroovyCompiler.execute(ApiGroovyCompiler.java:56) at org.gradle.api.internal.tasks.compile.daemon.CompilerDaemonServer.execute(CompilerDaemonServer.java:53) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at org.gradle.messaging.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:360) at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54) at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)

BUILD FAILED

Am I working on a wrong direction?

Thank you very much.

Regards, Yang

Posted by pjr (View)
Hi Yang,

I think there is a good path for you if you use the Gradle NetKernel plugin.

I suggest you prepare a configured NK instance locally, freeze it and then deploy (thaw) the frozen instance to your restricted production environment.

The doc section on freeze/thaw is most relevant.

The one thing you need to be able to do is either to be able to access an external maven repo with the frozen instances in it. Or alternatively, to use scp or rsync over ssh to sync the maven repo to the deployment machine and use it locally.

Take a look at those docs and let me know if this makes sense?

Cheers,

Peter

Like · Post Reply
nk4um Moderator
Posts: 901
June 25, 2015 12:30

Hi Yang,

I think there is a good path for you if you use the Gradle NetKernel plugin.

I suggest you prepare a configured NK instance locally, freeze it and then deploy (thaw) the frozen instance to your restricted production environment.

The doc section on freeze/thaw is most relevant.

The one thing you need to be able to do is either to be able to access an external maven repo with the frozen instances in it. Or alternatively, to use scp or rsync over ssh to sync the maven repo to the deployment machine and use it locally.

Take a look at those docs and let me know if this makes sense?

Cheers,

Peter

Like · Post Reply
nk4um User
Posts: 27
June 25, 2015 09:08update and deploy modules in an isolated network environment

Hi Peter,

I am trying to deploy Netkernel to an isolated network. I need to install and update some official modules but I cannot access to Netkernel central repository. Since all the modules exist as jar files so I thought maybe I can simply replace the ones in lib and modules directory that need to be updated, and correspondingly update etc/modules.xml. I am not very sure about it so I am writing to you to consult the feasibility and security of my idea. How do you think about it?

Thank you very much and have a nice day.

Regards, Yang

Like · Post Reply