wulfcrona
10/22/2021, 9:25 AMdatajoely
10/22/2021, 9:27 AMutf-16
or preferably, move to ParquetDataSet
we encourage that all data stored in the intermediate layer is typed rather than plan text like CSVwulfcrona
10/22/2021, 9:34 AMdatajoely
10/22/2021, 9:55 AMlatin-1
or potentially ISO-8859-1
as encoding options as per
https://stackoverflow.com/questions/5552555/unicodedecodeerror-invalid-continuation-bytewulfcrona
10/22/2021, 11:24 AMdatajoely
10/22/2021, 11:30 AMNC
10/30/2021, 12:16 AMkedro-docker
plugin, but cannot make the steps outlined in the Github repo tutorial.NC
10/30/2021, 12:17 AMkedro docker init
, I get the error message Error: No such command 'docker'.
NC
10/30/2021, 12:17 AMkedro docker
does not seem to be a valid command?NC
10/30/2021, 12:19 AMkedro-docker
plugin still being active developed? I would love to explore further using Kedro and Docker together, to facilitate pipeline sharing within my team and deployment to the production server.Zemeio
10/30/2021, 1:48 PMdocker-compose run --rm kedro-app kedro run
.
Ofc you would need to personalize it to work with viz and stuff.Zemeio
10/30/2021, 2:43 PMkedro docker init
, and it ran successfully. Are you sure you were using the right environment?NC
10/31/2021, 12:38 AMZemeio
10/31/2021, 12:46 AMZemeio
10/31/2021, 12:50 AMpip show kedro-docker
, do you get something like this?NC
10/31/2021, 3:53 AMkedro
commands in my virtual env, it’s still calling the Kedro installed in my global environment. When I made sure that I’m calling Kedro installed in the same virtual env as kedro-docker
, it is now working.Zemeio
10/31/2021, 5:15 AMNC
10/31/2021, 1:27 PMdatajoely
10/31/2021, 3:27 PMNC
10/31/2021, 4:33 PMdatajoely
10/31/2021, 7:34 PMNC
10/31/2021, 9:27 PMIsaac89
11/03/2021, 1:52 PMdatajoely
11/03/2021, 1:53 PMIsaac89
11/03/2021, 2:24 PMdatajoely
11/03/2021, 2:24 PMZhee
11/04/2021, 1:37 PMdatajoely
11/04/2021, 6:57 PMdatajoely
11/05/2021, 10:22 AMkedro-viz
to have some sort of 'catalog manager' where this would make sense to live. It's not under active development, but if users start shouting that they'd like it has more weight on the backlog 🙂
- Finally, the most structured way of doing this today is to use Sphinx and the built in kedro build-docs
command to generate static docs. This is mostly there for Python API docs, but everything on the Kedro docs (https://kedro.readthedocs.io/en/stable/) is made this way so you can steal how we do it too. I think it would be pretty neat to write a script that used the DataCatalog python API to create data documentation stubs which you then fill in with human readable descriptions.