Did you just overwrite the source code? It's fine if you are just doing it yourself. 1. Ideally if you can just upgrade to

0.18.2

, you can share the errors that you get and we can try work together and help you with the migration. 2. If upgrade is not possible, a slightly better option is make your own

CustomHiveDataSet

, that just inherit from the original

SparkHiveDataSet

and you implement the

_exists

method to override its behavior, you can package that in your project. One addition thing you need to is update the

DataSet

type in your

catalog,yml

to point to your custom implementation instead.