问题描述
使用了gradle编译插件,编译插件使用的是Transform处理字节码,如果第一次ctrl+c中断或者其它原因中断,下次再次构建会出现build文件夹清理不了的问题
Execution failed for task ':my-module:my-submodule:clean'.
> java.io.IOException: Unable to delete directory 'C:\Dev\myproject\my-module\my-submodule\build'Failed to delete some children. This might happen because a process has files open or has its working directory set in the target directory.- C:\Dev\myproject\my-module\my-submodule\build\libs\my-module.my-submodule.jar- C:\Dev\myproject\my-module\my-submodule\build\libs
相似问题
https://github.com/gradle/gradle/issues/26912
gradle的github项目上有人反馈这个问题,但不是gradle的问题,是编译插件的问题。
复现问题
- 使用gradle命令构建app
gradle assembledebug
- 进入transfrom的taskAction方法后,终止命令,注意不要强制杀死进程
- 马上执行下一个构建任务
gradle clean
gradle assembledebug
编写了一段java程序,用于模拟复现和修复问题:
import java.io.BufferedReader;
import java.io.File;
import java.lang.Thread;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;/*** 运行前,需要确保系统配置了正确的gradle版本、java版本* 监控Gradle命令执行并在检测到特定字符串后终止并重试* 注意::不能放到unitTest或者androidTest文件夹里面作为单元测试,会有gradle的影响* 需要作为java程序单独使用java命令执行mian函数* 1.编译:javac GradleTaskKillAndRebuildTest.java* 2.运行:java GradleTaskKillAndRebuildTest*/
public class GradleTaskKillAndRebuildTest {private static final String GRADLE_COMMAND = "gradle assembledebug";//查找日志,适当时机模拟退出private static final String TARGET_STRING = "[Plugin]";private static final String TARGET_STRING2 = "transform started";public static void main(String[] args) throws Exception {File path = new File("");String exitDaemonCommand = "cmd /c gradle --stop";String cleanCommand = "cmd /c cd " + path.getAbsolutePath() + " && gradle clean";String buildCommand = "cmd /c cd " + path.getAbsolutePath() + " && " + GRADLE_COMMAND;// 初始,先退出之前的后台进程executeCommand(true, exitDaemonCommand);// 删除build目录File buildDir = new File("./app/build");System.out.println(buildDir.getAbsolutePath());deleteBuildDirectory(buildDir);// 第一次构建:清理+构建中断executeCommand(true, cleanCommand);boolean foundAndExit = execAndFoundAndKill(buildCommand);if (!foundAndExit) {System.err.println("执行并查找线程启动标识失败");System.exit(2);return;}//等前面的执行完成Thread.sleep(5000L);// 第二次构建:清理+正常构建Pair<Boolean, String> cleanResult = executeCommand(true, cleanCommand);if (!cleanResult.first) {System.err.println("第二次构建清理失败");System.exit(41);return;}Pair<Boolean, String> checkResult = executeCommand(true, buildCommand);System.out.println("测试结果:" + checkResult.first);System.exit(checkResult.first ? 0 : 1);}/*** 执行命令并返回输出结果*/private static Pair<Boolean, String> executeCommand(boolean print, String command) {Process process;try {process = Runtime.getRuntime().exec(command);} catch (IOException e) {System.err.println("执行命令失败: " + e.getMessage());e.printStackTrace();return new Pair<>(false, "");}ExecutorService executor = Executors.newSingleThreadExecutor();StringBuilder output = new StringBuilder();Runnable task1 = () -> {// 读取标准输出try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()))) {String line = reader.readLine();while (line != null) {if (print) {System.out.println(line);}output.append(line).append(System.lineSeparator());line = reader.readLine();}} catch (IOException e) {System.err.println("读取标准输出时出错: " + e.getMessage());}};Runnable task2 = () -> {// 读取错误输出try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getErrorStream()))) {String line = reader.readLine();while (line != null) {if (print) {System.out.println(line); // 打印输出行}output.append(line).append(System.lineSeparator());line = reader.readLine();}} catch (IOException e) {System.err.println("读取错误输出时出错: " + e.getMessage());}};executor.submit(task1);executor.submit(task2);executor.shutdown();try {executor.awaitTermination(1, TimeUnit.HOURS); // 等待足够长的时间让任务完成int ret = process.waitFor();return new Pair<>(ret == 0, output.toString());} catch (InterruptedException e) {Thread.currentThread().interrupt();System.err.println("等待进程完成时被中断: " + e.getMessage());return new Pair<>(false, output.toString());}}/*** 执行gradle命令,找到特点字符,并终止,返回true则执行正常*/private static boolean execAndFoundAndKill(String gradleCommand) {System.out.println("执行Gradle命令 " + gradleCommand);try {Process process = Runtime.getRuntime().exec(gradleCommand);ExecutorService executor = Executors.newSingleThreadExecutor();// 启动线程读取输出并监控目标字符串boolean targetFound = monitorProcessOutput(process, executor);if (targetFound) {System.out.println("发现目标字符串: " + TARGET_STRING + ",正在终止进程...");process.destroy();if (process.waitFor(5, TimeUnit.SECONDS)) {System.out.println("进程已成功终止");} else {System.out.println("强制终止进程");process.destroyForcibly();}return true;} else {// 如果没有找到目标字符串,等待进程自然结束int exitCode = process.waitFor();System.out.println("进程正常结束,退出码: " + exitCode);}} catch (Exception e) {System.err.println("执行命令时出错: " + e.getMessage());e.printStackTrace();}return false;}/*** 监控进程输出,查找目标字符串* @param process 要监控的进程* @param executor 线程池用于异步读取输出* @return 是否找到目标字符串* @throws InterruptedException 线程中断异常*/private static boolean monitorProcessOutput(Process process, ExecutorService executor) throws InterruptedException {AtomicBoolean targetFound = new AtomicBoolean(false);Object lock = new Object();boolean shouldContinueMonitoring = true;// 读取标准输出executor.submit(() -> readStream(process.getInputStream(), targetFound, lock));// 读取错误输出executor.submit(() -> readStream(process.getErrorStream(), targetFound, lock));// 定期检查是否找到目标字符串while (shouldContinueMonitoring && !targetFound.get()) {synchronized (lock) {if (targetFound.get()) {shouldContinueMonitoring = false;} else {try {lock.wait(100); // 每100ms检查一次} catch (InterruptedException e) {Thread.currentThread().interrupt();shouldContinueMonitoring = false;}}}// 检查进程是否已经结束if (shouldContinueMonitoring) {try {if (process.waitFor(10, TimeUnit.MILLISECONDS)) {shouldContinueMonitoring = false; // 进程已结束}} catch (InterruptedException e) {Thread.currentThread().interrupt();shouldContinueMonitoring = false;}}}executor.shutdown();executor.awaitTermination(2, TimeUnit.SECONDS);return targetFound.get();}/*** 读取输入流并检查目标字符串*/private static void readStream(InputStream inputStream,AtomicBoolean targetFound,Object lock) {boolean found = false;int foundNextStep = 0;try {try (BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream))) {String line = reader.readLine();while (line != null && !targetFound.get()) {System.out.println(line); // 打印输出行if (line.contains(TARGET_STRING) && line.contains(TARGET_STRING2)) {found = true;}if (found) {foundNextStep++;}if (foundNextStep >= 4) {synchronized (lock) {targetFound.set(true);lock.notifyAll();}}line = reader.readLine();}}} catch (Exception e) {System.err.println("读取流时出错: " + e.getMessage());}}/*** 删除当前目录下的build目录*/private static void deleteBuildDirectory(File buildDir) {if (buildDir.exists() && buildDir.isDirectory()) {System.out.println("正在删除build目录...");if (deleteDirectoryRecursively(buildDir)) {System.out.println("build目录删除成功");} else {System.out.println("build目录删除失败");}} else {System.out.println("build目录不存在,无需删除");}}/*** 递归删除目录及其内容*/private static boolean deleteDirectoryRecursively(File dir) {// 先删除所有子文件和子目录File[] files = dir.listFiles();if (files != null) {for (File file : files) {if (file.isDirectory()) {deleteDirectoryRecursively(file);} else {// System.out.println("删除文件:" + file.getAbsolutePath());file.delete();}}}// 删除空目录return dir.delete();}/*** 简单的Pair类,用于同时返回两个值*/private static class Pair<T1, T2> {public final T1 first;public final T2 second;public Pair(T1 first, T2 second) {this.first = first;this.second = second;}}
}
产生原因
终止gradle任务,不一定是强制杀死jvm进程,而是等待jvm全部用户线程(isDaemon = false)结束(如:jenkins终止gradle打包任务时)
由于transform的操作属于用户线程,所以终止任务后,仍然需要等待transform执行完成。
执行clean时,使用
gradle --status
可以看到,一共有两个进程在busy工作状态,一个是clean任务,第二个是已发出停止但未真正停止的进程
未停止的进程还在占用build文件夹、transform输出的classes文件,所以不能被clean任务删除
验证
打印日志到文件,注意加上pid
- 监听gradle构建事件buildFinished,将构建结束事件打印日志到文件
- transform开始事件打印日志到文件
- 监听gradle构建事件projectsLoaded,将构建解析完成事件打印到日志文件
经过上述“复现问题”的步骤,你会发现下面类似的日志:
- gradle assembledebug(pid=1) => projectsLoaded
- gradle assembledebug (pid=1) => transform started
- gradle clean (pid=2) => projectsLoaded
- gradle clean (pid=2) => build failure: java.io.IOException: Unable to delete directory xxxx\build
- gradle assembledebug(pid=1) => buildFinished
解决方法
完善gradle插件的优雅退出
- 添加addShutdownHook监听退出事件,回收关闭资源
Runtime.getRuntime().addShutdownHook(new Thread(() -> {// 这里会被回调System.err.println("收到优雅停止信号,开始收尾...");// 1. 释放资源、刷日志、关闭连接// 2. 记录状态、删除临时文件System.err.println("收尾完成,JVM 即将退出");}));
- 将非必要等待的线程,标记为守护线程(非用户线程不会被等待结束)
Thread thread = new Thread();thread.isDaemon = truethread .start()
强制杀掉jvm进程
- transform开始时,记录当前进程pid,写到一个固定的文件(用于标记构建进行中)
DXGradlePlugin.pidFile = File("./myBuild.pid")
val pid = ProcessHandle.current().pid()
DXGradlePlugin.pidFile.writeText("$pid")
- buildFinished事件(构建完成后)删除pid文件(标记构建完成)
DXGradlePlugin.pidFile.delete()
3.每次启动时(如projectsEvaluated事件配置完成后)检测未完成的pid,等待或强制杀死进程
//gradle配置完成
override fun projectsEvaluated(gradle: Gradle) {val startWaitTime = System.currentTimeMillis()var hasTryKill = falsewhile (true) {if (!flagFile.exists()) {//标记文件不存在break}if (!hasTryKill && System.currentTimeMillis() - startWaitTime >= 10000) {//等10秒后,尝试杀掉进程hasTryKill = trueval pid = flagFile.readText()// kill busy pid:$pid"val command = "cmd /c taskkill /PID $pid /T /F"var ret = Runtime.getRuntime().exec(command).waitFor()//标记文件不存在if (ret == 0) {flagFile.delete()} else {val command2 = "kill -9 $pid"ret = Runtime.getRuntime().exec(command2).waitFor()}// "kill busy pid ret:$ret")if (ret == 0) {flagFile.delete()break}}// "wait other running take(exit task)")Thread.sleep(1000)}if (System.currentTimeMillis() - startWaitTime >= 60000) {//等待超时,1分钟//"wait other running take time out")break}}
修复结果
修改完成后,重新执行:
java GradleTaskKillAndRebuildTest
清理成功:
最终测试成功: