Bintray with sbt "addCompilerPlugin"

Aliceravier · October 27, 2020, 3:32pm

Hi,

I have published an sbt scala project (a compiler plugin + library) to Bintray and I’m not sure how to add it into another sbt project with “addCompilerPlugin”. I can’t seem to find proper explanations of what should be included in the “addCompilerPlugin” command.
My plugin/library is up at https://bintray.com/aliceravier/maven/scala-mungo-prototype/1.0#.

I have tried adding:
resolvers += Resolver.bintrayRepo(“aliceravier”, “maven”)
autoCompilerPlugins := true
addCompilerPlugin(“com.bintray.api” %% “scala-mungo-prototype_2.13” % “1.0”)

in “build.sbt”, but sbt cannot extract the project structure when I add the third line (addCompilerPlugin).
It gives me this output in the log file:

2020-10-27 13:52:46,866 [76622069] INFO - System.util.ExternalSystemUtil - External project [C:/Year five/Project/OnlineTestPlugin] resolution task started
2020-10-27 13:52:59,801 [76635004] INFO - System.util.ExternalSystemUtil - External project [C:/Year five/Project/OnlineTestPlugin] resolution task executed in 12935 ms.
2020-10-27 13:53:00,166 [76635369] INFO - ge.ExternalProjectsDataStorage - Save external projects data in 20 ms
2020-10-27 13:53:07,527 [76642730] INFO - rationStore.ComponentStoreImpl - Saving Project(name=Scala-Mungo-Prototype, containerState=ACTIVE, componentStore=C:\Year five\Project\Scala-Mungo\Scala-Mungo-Prototype)RunManager took 11 ms
2020-10-27 13:53:18,966 [76654169] INFO - System.util.ExternalSystemUtil - External project [C:/Year five/Project/OnlineTestPlugin] resolution task started
2020-10-27 13:53:29,990 [76665193] INFO - dea.updater.SdkComponentSource - File C:\Users\Alice.android\repositories.cfg could not be loaded.
2020-10-27 13:53:29,992 [76665195] INFO - dea.updater.SdkComponentSource - Downloading https://dl.google.com/android/repository/addons_list-3.xml
2020-10-27 13:53:30,495 [76665698] INFO - dea.updater.SdkComponentSource - Downloading https://dl.google.com/android/repository/sys-img/android/sys-img2-1.xml
2020-10-27 13:53:30,495 [76665698] INFO - dea.updater.SdkComponentSource - Downloading https://dl.google.com/android/repository/repository2-1.xml
2020-10-27 13:53:30,495 [76665698] INFO - dea.updater.SdkComponentSource - Downloading https://dl.google.com/android/repository/sys-img/android-automotive/sys-img2-1.xml
2020-10-27 13:53:30,495 [76665698] INFO - dea.updater.SdkComponentSource - Downloading https://dl.google.com/android/repository/addon2-1.xml
2020-10-27 13:53:30,495 [76665698] INFO - dea.updater.SdkComponentSource - Downloading https://dl.google.com/android/repository/sys-img/android-tv/sys-img2-1.xml
2020-10-27 13:53:30,505 [76665708] INFO - dea.updater.SdkComponentSource - Downloading https://dl.google.com/android/repository/glass/addon2-1.xml
2020-10-27 13:53:30,505 [76665708] INFO - dea.updater.SdkComponentSource - Downloading https://dl.google.com/android/repository/sys-img/android-wear/sys-img2-1.xml
2020-10-27 13:53:30,505 [76665708] INFO - dea.updater.SdkComponentSource - Downloading https://dl.google.com/android/repository/sys-img/android-wear-cn/sys-img2-1.xml
2020-10-27 13:53:30,505 [76665708] INFO - dea.updater.SdkComponentSource - Downloading https://dl.google.com/android/repository/sys-img/google_apis_playstore/sys-img2-1.xml
2020-10-27 13:53:30,505 [76665708] INFO - dea.updater.SdkComponentSource - Downloading https://dl.google.com/android/repository/extras/intel/addon2-1.xml
2020-10-27 13:53:30,505 [76665708] INFO - dea.updater.SdkComponentSource - Downloading https://dl.google.com/android/repository/sys-img/google_apis/sys-img2-1.xml
2020-10-27 13:53:30,851 [76666054] INFO - System.util.ExternalSystemUtil - External project [C:/Year five/Project/OnlineTestPlugin] resolution task executed in 11885 ms.
2020-10-27 13:53:31,186 [76666389] INFO - ge.ExternalProjectsDataStorage - Save external projects data in 10 ms
2020-10-27 13:53:38,866 [76674069] INFO - System.util.ExternalSystemUtil - External project [C:/Year five/Project/OnlineTestPlugin] resolution task started
2020-10-27 13:54:04,086 [76699289] INFO - System.util.ExternalSystemUtil - External project [C:/Year five/Project/OnlineTestPlugin] resolution task executed in 25220 ms.
2020-10-27 13:54:04,425 [76699628] INFO - ge.ExternalProjectsDataStorage - Save external projects data in 0 ms
2020-10-27 13:54:32,455 [76727658] INFO - System.util.ExternalSystemUtil - External project [C:/Year five/Project/OnlineTestPlugin] resolution task started
2020-10-27 13:54:34,661 [76729864] INFO - rationStore.ComponentStoreImpl - Saving appEditorSettings took 15 ms
2020-10-27 13:54:45,145 [76740348] INFO - System.util.ExternalSystemUtil - External project [C:/Year five/Project/OnlineTestPlugin] resolution task executed in 12690 ms.
2020-10-27 13:54:45,605 [76740808] INFO - ge.ExternalProjectsDataStorage - Save external projects data in 90 ms
2020-10-27 13:55:32,795 [76787998] INFO - System.util.ExternalSystemUtil - External project [C:/Year five/Project/OnlineTestPlugin] resolution task started
2020-10-27 13:55:45,665 [76800868] INFO - System.util.ExternalSystemUtil - External project [C:/Year five/Project/OnlineTestPlugin] resolution task executed in 12870 ms.
2020-10-27 13:55:46,015 [76801218] INFO - ge.ExternalProjectsDataStorage - Save external projects data in 10 ms
2020-10-27 14:21:56,199 [78371402] INFO - System.util.ExternalSystemUtil - External project [C:/Year five/Project/OnlineTestPlugin] resolution task started
2020-10-27 14:22:13,903 [78389106] INFO - System.util.ExternalSystemUtil - External project [C:/Year five/Project/OnlineTestPlugin] resolution task executed in 17704 ms.
2020-10-27 14:22:14,258 [78389461] INFO - ge.ExternalProjectsDataStorage - Save external projects data in 10 ms
2020-10-27 14:22:45,738 [78420941] INFO - .script.IdeScriptEngineManager - javax.script.ScriptEngineManager initialized in 259 ms
2020-10-27 14:23:00,358 [78435561] INFO - j.ide.actions.RevealFileAction - Exit code 1
2020-10-27 14:26:32,547 [78647750] INFO - j.ide.actions.RevealFileAction - Exit code 1

SethTisue · October 27, 2020, 5:36pm

Try this:

addCompilerPlugin("default" %% "scala-mungo-prototype" % "1.0")

I made two changes:

If you use %%, you mustn’t also include _2.13, the purpose of %% is to add that for you
Not sure why you thought the plugin had been published under the com.bintray.api organization, but in any case, I found it by visiting https://dl.bintray.com/aliceravier/maven/ and seeing that the only directory present is named default

Aliceravier · October 28, 2020, 11:04am

Thanks! That worked.
The problem was that I didn’t understand how addCompilerPlugin worked.
I assume that it is (at least for bintray):
addCompilerPlugin(organisationName %% packageName % versionNumber)

SethTisue · October 28, 2020, 3:07pm

I guess your project is about done. Is there anything that’s missing from https://docs.scala-lang.org/overviews/plugins/index.html that we ought to add, that would have helped you to have known…?

Aliceravier · October 30, 2020, 4:07pm

Hi,

Well, I might still need to deal with break statements which probably means making my own control flow graph (unless there is a way of accessing one that scalac generates?). I hadn’t realised I really needed that before publishing…

Yes! I have made a list of things which would be useful to have, in my opinion:

It would have really helped to get a link to the quasiquotes page: https://docs.scala-lang.org/overviews/quasiquotes/syntax-summary.html
with an explanation that these are useful for pattern matching, and an example.

This doesn’t seem to be true anymore:
“”"
Note however that addCompilerPlugin only adds the JAR to the compilation classpath;
it doesn’t actually enable the plugin.
To do that, you must customize scalacOptions to include the appropriate -Xplugin call.
To shield users from having to know this,
it’s relatively common for compiler plugin authors to also write an accompanying sbt plugin
that takes of customizing the classpath and compiler options appropriately.
Then using your plugin only requires adding an addSbtPlugin(…) call to project/plugins.sbt.
“”"
so removing that would be good.
Same for:
“”"
To use the plugin, a user adds the JAR file to their compile-time classpath and enables it by invoking scalac with -Xplugin:…
“”"

Also, this is I guess more of a problem with Intellij, but it kept highlighting my correct plugin code in red which was very confusing.
It would be nice if the page warned that some editors do this and that one should try compiling it anyway.
For me it was mostly having problems with Trees#Tree, List[List[ValDef]] and Seq[Trees#Tree], and I ended up just supressing the red highlighting.

It would have been helpful to get more of an explanation of how the internals of the division by zero example plugin worked.
By the internals, I mean this part:

for ( tree @ Apply(Select(rcvr, nme.DIV), List(Literal(Constant(0)))) <- unit.body
             if rcvr.tpe <:< definitions.IntClass.tpe)
          {
            global.reporter.error(tree.pos, "definitely division by zero")
          }

Explaining that it looks through the (abstract synthax tree of) code for methods and for each one checks if it is division by zero.
(Adding the link to the quasiquotes page here might be good as they are a different way to pattern match)

Also, it wasn’t clear to me that the xml file actually
needs to be in the “src\main\scala” directory for the publishing to work, so mentioning that would be good.
(I originally thought that the xml file was only needed for creating the jar locally and that the publishing would take that jar instead of what was inside my src folder, which admittedly was very foolish of me)

It would be great if
“”"
When you are happy with how the plugin behaves, you may wish to publish the JAR to a Maven or Ivy repository where it can be resolved by a build tool.
“”"
had more information on how to publish a project, or linked to some tutorials.
Knowing how addCompilerPlugin() worked (what arguments it takes) would have been very useful.

Also the talk you did was quite helpful for me as an introduction to compiler plugins, so it would be nice to have on the page, if that’s possible:

Thanks for your help!

SethTisue · November 11, 2020, 9:44pm

Thank you, that’s excellent feedback! I will use this to make some improvements to the page. (But I probably won’t get to it until after we finish releasing Scala 2.13.4.)

SethTisue · November 18, 2020, 12:27am

Here’s a pull request where I took nearly all of your suggestions https://github.com/scala/docs.scala-lang/pull/1819

@Aliceravier if you could take a look, that’d be awesome

Aliceravier · November 18, 2020, 10:26am

That looks great! Thanks for taking advice about this

The lines

That's how it works with no build tool. If you are using sbt to build

your plugin, then the XML file goes in `src/main/resources`.

might be confusing. At least when I am developping the plugin, I still use the -Xplugin command when testing (even though I am using sbt), as opposed to publishing it anew each time. So it might be good to clarify that the jar method is to be used before publication and having the xml file in src/main/resources is useful for publishing the plugin.

Also, would it be worth having information for build tools other than sbt? I don’t know if they work very similarly.

SethTisue · November 18, 2020, 6:24pm

re: the XML file, when doing local development, you don’t actually need to publishLocal, but you can package, which will produce a JAR in the target directory that includes the thing from src/main/resources. I would consider that standard, but the approach you used is of course workable as well. idk, I think it might be a bit too much nuance to get into on the page.

re: other build tools than sbt, somebody else would have to write that, I only use sbt

Aliceravier · November 18, 2020, 7:09pm

Fair, I had never used Scala before so I probably came into this with less experience than the average person.
Sounds good to me :))

SethTisue · November 18, 2020, 7:48pm

I had never used Scala before

talk about jumping in the deep end!

Aliceravier · November 18, 2020, 7:58pm

Yeah, supervisors expect a lot from students It was fun though, I like Scala.