GDExporter cannot find GDevelop Core

Is GDExporter still a supported component of the GDevelop experience?

Quite some time ago I added gd-exporter into my ci-cd pipeline. Recently the pipeline fails with this output, and I guess the fix is to change permissions on the published GDevelop installation.

+ /usr/local/lib/node_modules/gdexporter/bin/cli -p /project/game.json -o /output
πŸ““ No config file found!
βŒ› Loading plugins...
βŒ› Loading GDCore...
πŸ•— Getting latest release tag...
❌ The GDevelop version was not found, downloading it!
πŸ•— Starting download of GDevelop Runtime 'v5.0.132'...
πŸ•— Starting download of GDevelop Core...
/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/downloadGD.js:12
          throw new Error(
          ^

Error: ❌ Cannot download https://s3.amazonaws.com/gdevelop-gdevelop.js/master/commit/bb3abdb1fa85a2d9e89e5053ae47deef1008299a/libGD.js! Error 403: Forbidden
    at RedirectableRequest.<anonymous> (/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/downloadGD.js:12:17)
    at RedirectableRequest.emit (node:events:527:28)
    at RedirectableRequest._processResponse (/usr/local/lib/node_modules/gdexporter/node_modules/follow-redirects/index.js:346:10)
    at ClientRequest.RedirectableRequest._onNativeResponse (/usr/local/lib/node_modules/gdexporter/node_modules/follow-redirects/index.js:57:10)
    at Object.onceWrapper (node:events:642:26)
    at ClientRequest.emit (node:events:527:28)
    at HTTPParser.parserOnIncomingClient [as onIncoming] (node:_http_client:631:27)
    at HTTPParser.parserOnHeadersComplete (node:_http_common:128:17)
    at TLSSocket.socketOnData (node:_http_client:494:22)
    at TLSSocket.emit (node:events:527:28)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at TLSSocket.Readable.push (node:internal/streams/readable:234:10)
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23)

Node.js v17.9.0
+ exit 1

Well, I’m not sure if it has ever really been a component of GDevelop, but let’s ask the author.
@arthuro555, are you maintaining GDExporter? Is it possible that something got… outdated? See OP.

GDexporter is and always was a community project, not a part of the GDevelop Company’s official ecosystem.

This error is quite odd and I’ll investigate it later. That URL is part of the GDevelop infrastructure where I fetch prebuilt binaries from. As far as I am aware, this should still be existing as it is as far as I recall also in use in the GDevelop repository

Yes, I still maintain and use GDexporter, and will update it if it has to be.

2 Likes

I was not aware there is a company behind GDevelop.
But yes, if that URL is part of GDevelop infrastructure it may need a fix anyway.

The GDExporter is doing a marvellous job - I am really happy about it. :slight_smile:

That’s understandable, it has been founded not too long ago for the maintainer and a few other people to get to work full-time on GDevelop.

I’m glad it can be useful to you :+1:

So, I just took a look and think I pinpointed the issue. The commit linked to this release is the following one:

As you can see, it contains a [skip ci] tag, which prevented the CI responsible for building and uploading this library from running for this commit.
I’ve fixed this on version 3.2.7, by making it use the nearest parent commit without the skip-ci tag.

Good match. My last successful run was on APR22, the first failure on APR30.
For the root cause I am not sure I can follow. But I can let you know once it works again. :slight_smile:

Just now it still failed. BTW, version number 3.2.7 refers to GDExporter, doesn’t it? Does GDExporter also output that version? So far I am sure to run the latest available version but cannot tell exactly which one it is…

+ /usr/local/lib/node_modules/gdexporter/bin/cli -p /project/game.json -o /output
πŸ““ No config file found!
βŒ› Loading plugins...
βŒ› Loading GDCore...
πŸ•— Getting latest release tag...
❌ The GDevelop version was not found, downloading it!
πŸ•— Starting download of GDevelop Runtime 'v5.0.132'...
πŸ•— Starting download of GDevelop Core...
/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/downloadGD.js:12
          throw new Error(
          ^

Error: ❌ Cannot download https://s3.amazonaws.com/gdevelop-gdevelop.js/master/commit/bb3abdb1fa85a2d9e89e5053ae47deef1008299a/libGD.js! Error 403: Forbidden
    at RedirectableRequest.<anonymous> (/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/downloadGD.js:12:17)
    at RedirectableRequest.emit (node:events:527:28)
    at RedirectableRequest._processResponse (/usr/local/lib/node_modules/gdexporter/node_modules/follow-redirects/index.js:346:10)
    at ClientRequest.RedirectableRequest._onNativeResponse (/usr/local/lib/node_modules/gdexporter/node_modules/follow-redirects/index.js:57:10)
    at Object.onceWrapper (node:events:642:26)
    at ClientRequest.emit (node:events:527:28)
    at HTTPParser.parserOnIncomingClient [as onIncoming] (node:_http_client:631:27)
    at HTTPParser.parserOnHeadersComplete (node:_http_common:128:17)
    at TLSSocket.socketOnData (node:_http_client:494:22)
    at TLSSocket.emit (node:events:527:28)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at TLSSocket.Readable.push (node:internal/streams/readable:234:10)
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23)

Node.js v17.9.0
+ exit 1

So something indeed changed. Trying to compile again I have a new error. I am not sure whether it is connected or a different issue, but I’ll share it here:

TypeError: Failed to parse URL from /usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/Versions/v5.0.137/libGD.wasm
    at new Request (node:internal/deps/undici/undici:4813:19)
    at Agent.fetch2 (node:internal/deps/undici/undici:5505:29)
    ... 6 lines matching cause stack trace ...
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Emitted 'error' event on WrappedGD instance at:
    at Object.onAbort (/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/WrappedGD.js:51:28)
    at process.abort (/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/Versions/v5.0.137/libGD.js:9:15232)
    at process.emit (node:events:527:28)
    at emit (node:internal/process/promises:149:20)
    at processPromiseRejections (node:internal/process/promises:283:27)
    at process.processTicksAndRejections (node:internal/process/task_queues:96:32) {
  [cause]: TypeError [ERR_INVALID_URL]: Invalid URL
      at new NodeError (node:internal/errors:377:5)
      at URL.onParseError (node:internal/url:563:9)
      at new URL (node:internal/url:643:5)
      at new Request (node:internal/deps/undici/undici:4811:25)
      at Agent.fetch2 (node:internal/deps/undici/undici:5505:29)
      at Object.fetch (node:internal/deps/undici/undici:6332:20)
      at fetch (node:internal/bootstrap/pre_execution:196:25)
      at instantiateAsync (/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/Versions/v5.0.137/libGD.js:9:17055)
      at createWasm (/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/Versions/v5.0.137/libGD.js:9:17655)
      at /usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/Versions/v5.0.137/libGD.js:9:95296
      at getGD (/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/downloadGD.js:208:57)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
    input: '/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/Versions/v5.0.137/libGD.wasm',
    code: 'ERR_INVALID_URL'
  }
}

:thinking: try deleting node_modules/gdcore-tools and reinstalling gdexporter

Thank you for the hint. Actually I never delete stuff - I build from scratch. My pipeline triggers automatically once a month. It builds a docker container for gdexporter on it’s own using this dockerfile:

FROM node

RUN mkdir /project
RUN mkdir /output
RUN npm i -g gdexporter

COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
#CMD [ "-p", "/project", "-o", "/output" ]

VOLUME ["/usr/local/lib/node_modules/gdexporter/node_modules"]

With this container freshly created I then compiled my projects and got this output:

TypeError: Failed to parse URL from /usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/Versions/v5.0.138/libGD.wasm
    at new Request (node:internal/deps/undici/undici:4832:19)
    at Agent.fetch2 (node:internal/deps/undici/undici:5524:29)
    ... 6 lines matching cause stack trace ...
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Emitted 'error' event on WrappedGD instance at:
    at Object.onAbort (/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/WrappedGD.js:51:28)
    at process.abort (/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/Versions/v5.0.138/libGD.js:9:15232)
    at process.emit (node:events:537:28)
    at emit (node:internal/process/promises:149:20)
    at processPromiseRejections (node:internal/process/promises:283:27)
    at process.processTicksAndRejections (node:internal/process/task_queues:96:32) {
  [cause]: TypeError [ERR_INVALID_URL]: Invalid URL
      at new NodeError (node:internal/errors:388:5)
      at URL.onParseError (node:internal/url:564:9)
      at new URL (node:internal/url:644:5)
      at new Request (node:internal/deps/undici/undici:4830:25)
      at Agent.fetch2 (node:internal/deps/undici/undici:5524:29)
      at Object.fetch (node:internal/deps/undici/undici:6351:20)
      at fetch (node:internal/bootstrap/pre_execution:197:25)
      at instantiateAsync (/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/Versions/v5.0.138/libGD.js:9:17055)
      at createWasm (/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/Versions/v5.0.138/libGD.js:9:17655)
      at /usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/Versions/v5.0.138/libGD.js:9:95296
      at getGD (/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/downloadGD.js:208:57)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
    input: '/usr/local/lib/node_modules/gdexporter/node_modules/gdcore-tools/src/Versions/v5.0.138/libGD.wasm',
    code: 'ERR_INVALID_URL'
  }
}

Node.js v18.4.0

This time I added one more line of output, displaying the node version number.
It seems to be that apart from the gdexporter version number nothing changed.

Oh, I see there is a reference to a docker volume that might contain the files as a cache. But the pipeline that compiles the gdevelop projects (there are a handful) creates the volume, runs gdexporter for each project and then removes the volume again. Thus even if that volume would contain wrong files - it gets recreated from scatch every time I compile.

Hence I believe it is not a caching issue.

:thinking: I think I use an older LTS version of node, either v14 or v16. Maybe there was a breaking change in modules resolution causing this to happen?

1 Like

Indeed, that fixed it. I just changed the first line in Dockerfile to
FROM node:lts

Now the whole setup is based on node 16.15.1 and the error is gone. :slight_smile:

1 Like