A file is read from an excel sheet, transformed, and this is the output of that transformation. The issue occurs when the downstream node tries to open the file. The problem is with the swedish character Ö.

Okay two options - use

utf-16

or preferably, move to

ParquetDataSet

we encourage that all data stored in the intermediate layer is typed rather than plan text like CSV

Had no luck with utf-16, and not too familiar with ParquetDataSet. I'll do an ascii encode/decode workaround for now and have that tech debt for sunday. Thanks a lot for your help!

It's very confusing - only other thing I can suggest is to type

latin-1

or potentially

ISO-8859-1

as encoding options as per https://stackoverflow.com/questions/5552555/unicodedecodeerror-invalid-continuation-byte

I fixed it the issue was open_args_save. Looking at the fsspec source the encoding for fsspec.open defaults to "utf8" and breaks while passing "utf-8" in catalog works. Very odd, not sure if its a local "feature" for me or something larger.

Very weird - but glad it's fixed!

Hello! I have been trying to use the

kedro-docker

plugin, but cannot make the steps outlined in the Github repo tutorial.

When I try to run

kedro docker init

, I get the error message

Error: No such command 'docker'.

kedro docker

does not seem to be a valid command?

Is the

kedro-docker

plugin still being active developed? I would love to explore further using Kedro and Docker together, to facilitate pipeline sharing within my team and deployment to the production server.

Hello. I did not use kedro docker to create it, but I created a Dockerfile, a compose and a .dockerignore that seemed to work for me when I was testing some kedro stuff, maybe it can help you with something? https://gitlab.com/Zemeio/kedrotestproject You can run it by entering the kedro-experiments folder and running

docker-compose run --rm kedro-app kedro run

. Ofc you would need to personalize it to work with viz and stuff.

@User As for your question, I think that the kedro-docker is still the preferred way of doing it in general. I tried creating a new project, installing kedro-docker and running the

kedro docker init

, and it ran successfully. Are you sure you were using the right environment?

I did this in a fresh virtual env. What do you mean by “the right environment”?

Sometimes I run things and I realise I forgot to activate the environment, since I have Kedro on my base python and my venv.

When you run

pip show kedro-docker

, do you get something like this?

Thank you very much for the tip off! My print out looks exactly like yours, but this led me to realize that when I call

kedro

commands in my virtual env, it’s still calling the Kedro installed in my global environment. When I made sure that I’m calling Kedro installed in the same virtual env as

kedro-docker

, it is now working.

I'm glad you managed to solve your problem!

Thank you for pointing me in the right direction, Zemeio!

Amazing! Thanks for helping out @Zemeio you have been upgraded to status!

@NC did you starter a similar GitHub discussion? I was going to post a response tomorrow

I did post in the GitHub discussion before remembering that there is a Discord. Should I remove that post?

No worries - I’ll reply tomorrow so people arriving via search see the responses, but good to know that you at least have an answer. In short kedro docker is mostly for absolutely newbies, so the advise is going to be that you can always define your own container

Thank you! I went the route of defining my own Dockerfile for a Kedro pipeline and it seemed to be working fine, so definitely a totally feasible option for a newbie. 🙂

Hi ! I tried to use the TemplatedConfigLoader with jinja2 syntax in kedro 0.17.0 and 0.17.5 but the keys are not being substituted. Is the compatibility with jinja2 template already implemented ? Has anyone tried the jinja2 syntax within the catalog i.e. using {{  value_to_be_sobstituted }}?

There should be no change. Can you run the example? https://kedro.readthedocs.io/en/stable/04_kedro_project_setup/02_configuration.html#jinja2-support

yes it is working! I found the error. I set the globals_dict to an empty dict instead of None. Thanks for helping.

No worries @User

Hello Everyone. I have a question about documentation best practices. The autogenerated documentation is great to focus on pipelines and package but i would like also to document more my datasets. (I generated a static site on github pages) What would be the best approach to add details about datasources (meaning, content .. )? The data catalog could be a good place to start but the yaml file doesn't gives us access to description fields or other things like this (or maybe i missed something). (Or it's simply not the right place to write such piece of information.) Should i add details in a custom part of the sphinx project or could it be incorporated somewhere else ? What are you used to do when documenting your datasources?

I haven’t forgotten about this @Zhee ! I will write up a proper response tomorrow

Hi @User - now I can write up my thoughts now. tldr - out of the box we don't do all we could do in this area and I personally want to limit the YAML our users have to write. - Looking at the industry the best things I've seen are what Great Expectations (https://docs.greatexpectations.io/docs/tutorials/getting_started/check_out_data_docs/) and dbt (https://docs.getdbt.com/docs/building-a-dbt-project/documentation) are able to do. - Practically - it's not a lot of work to work with a Kedro DataCatalog object with the Python API, feed it into GE and generate this sort of documentation - it's not something we offer as a first party integration (yet) but would like to do some day. - Something we've been keen to do for a long time is extend

kedro-viz

to have some sort of 'catalog manager' where this would make sense to live. It's not under active development, but if users start shouting that they'd like it has more weight on the backlog 🙂 - Finally, the most structured way of doing this today is to use Sphinx and the built in

kedro build-docs

command to generate static docs. This is mostly there for Python API docs, but everything on the Kedro docs (https://kedro.readthedocs.io/en/stable/) is made this way so you can steal how we do it too. I think it would be pretty neat to write a script that used the DataCatalog python API to create data documentation stubs which you then fill in with human readable descriptions.