https://kedro.org/ logo
Join the conversationJoin Discord
Channels
advanced-need-help
announcements
beginners-need-help
introductions
job-posting
plugins-integrations
random
resources
welcome
Powered by Linen
beginners-need-help
  • d

    datajoely

    10/22/2021, 9:06 AM
    Hi @User - this is likely fixed by providing file system arguments like so:
  • w

    wulfcrona

    10/22/2021, 9:19 AM
    Tried to add the encoding in the fs_args and get the same error message preprocessed_MediaInvestment: type: pandas.CSVDataSet filepath: data/02_intermediate/preprocessed_MediaInvestment.csv save_args: encoding: 'utf-8' index: True load_args: encoding: 'utf-8' fs_args: open_args_load: encoding: 'utf-8' layer: intermediate
  • d

    datajoely

    10/22/2021, 9:22 AM
    How peculiar - to be clear is this file generated as part of a Kedro run or is it from something external?
  • w

    wulfcrona

    10/22/2021, 9:25 AM
    A file is read from an excel sheet, transformed, and this is the output of that transformation. The issue occurs when the downstream node tries to open the file. The problem is with the swedish character Ö.
  • d

    datajoely

    10/22/2021, 9:27 AM
    Okay two options - use
    utf-16
    or preferably, move to
    ParquetDataSet
    we encourage that all data stored in the intermediate layer is typed rather than plan text like CSV
  • w

    wulfcrona

    10/22/2021, 9:34 AM
    Had no luck with utf-16, and not too familiar with ParquetDataSet. I'll do an ascii encode/decode workaround for now and have that tech debt for sunday. Thanks a lot for your help!
  • d

    datajoely

    10/22/2021, 9:55 AM
    It's very confusing - only other thing I can suggest is to type
    latin-1
    or potentially
    ISO-8859-1
    as encoding options as per https://stackoverflow.com/questions/5552555/unicodedecodeerror-invalid-continuation-byte
  • w

    wulfcrona

    10/22/2021, 11:24 AM
    I fixed it the issue was open_args_save. Looking at the fsspec source the encoding for fsspec.open defaults to "utf8" and breaks while passing "utf-8" in catalog works. Very odd, not sure if its a local "feature" for me or something larger.
  • d

    datajoely

    10/22/2021, 11:30 AM
    Very weird - but glad it's fixed!
  • n

    NC

    10/30/2021, 12:16 AM
    Hello! I have been trying to use the
    kedro-docker
    plugin, but cannot make the steps outlined in the Github repo tutorial.
  • n

    NC

    10/30/2021, 12:17 AM
    When I try to run
    kedro docker init
    , I get the error message
    Error: No such command 'docker'.
  • n

    NC

    10/30/2021, 12:17 AM
    kedro docker
    does not seem to be a valid command?
  • n

    NC

    10/30/2021, 12:19 AM
    Is the
    kedro-docker
    plugin still being active developed? I would love to explore further using Kedro and Docker together, to facilitate pipeline sharing within my team and deployment to the production server.
  • z

    Zemeio

    10/30/2021, 1:48 PM
    Hello. I did not use kedro docker to create it, but I created a Dockerfile, a compose and a .dockerignore that seemed to work for me when I was testing some kedro stuff, maybe it can help you with something? https://gitlab.com/Zemeio/kedrotestproject You can run it by entering the kedro-experiments folder and running
    docker-compose run --rm kedro-app kedro run
    . Ofc you would need to personalize it to work with viz and stuff.
  • z

    Zemeio

    10/30/2021, 2:43 PM
    @User As for your question, I think that the kedro-docker is still the preferred way of doing it in general. I tried creating a new project, installing kedro-docker and running the
    kedro docker init
    , and it ran successfully. Are you sure you were using the right environment?
  • n

    NC

    10/31/2021, 12:38 AM
    I did this in a fresh virtual env. What do you mean by “the right environment”?
  • z

    Zemeio

    10/31/2021, 12:46 AM
    Sometimes I run things and I realise I forgot to activate the environment, since I have Kedro on my base python and my venv.
  • z

    Zemeio

    10/31/2021, 12:50 AM
    When you run
    pip show kedro-docker
    , do you get something like this?
  • n

    NC

    10/31/2021, 3:53 AM
    Thank you very much for the tip off! My print out looks exactly like yours, but this led me to realize that when I call
    kedro
    commands in my virtual env, it’s still calling the Kedro installed in my global environment. When I made sure that I’m calling Kedro installed in the same virtual env as
    kedro-docker
    , it is now working.
  • z

    Zemeio

    10/31/2021, 5:15 AM
    I'm glad you managed to solve your problem!
  • n

    NC

    10/31/2021, 1:27 PM
    Thank you for pointing me in the right direction, Zemeio!
  • d

    datajoely

    10/31/2021, 3:26 PM
    Amazing! Thanks for helping out @Zemeio you have been upgraded to status!
  • d

    datajoely

    10/31/2021, 3:27 PM
    @NC did you starter a similar GitHub discussion? I was going to post a response tomorrow
  • n

    NC

    10/31/2021, 4:33 PM
    I did post in the GitHub discussion before remembering that there is a Discord. Should I remove that post?
  • d

    datajoely

    10/31/2021, 7:34 PM
    No worries - I’ll reply tomorrow so people arriving via search see the responses, but good to know that you at least have an answer. In short kedro docker is mostly for absolutely newbies, so the advise is going to be that you can always define your own container
  • n

    NC

    10/31/2021, 9:27 PM
    Thank you! I went the route of defining my own Dockerfile for a Kedro pipeline and it seemed to be working fine, so definitely a totally feasible option for a newbie. 🙂
  • i

    Isaac89

    11/03/2021, 1:52 PM
    Hi ! I tried to use the TemplatedConfigLoader with jinja2 syntax in kedro 0.17.0 and 0.17.5 but the keys are not being substituted. Is the compatibility with jinja2 template already implemented ? Has anyone tried the jinja2 syntax within the catalog i.e. using {{  value_to_be_sobstituted }}?
  • d

    datajoely

    11/03/2021, 1:53 PM
    There should be no change. Can you run the example? https://kedro.readthedocs.io/en/stable/04_kedro_project_setup/02_configuration.html#jinja2-support
  • i

    Isaac89

    11/03/2021, 2:24 PM
    yes it is working! I found the error. I set the globals_dict to an empty dict instead of None. Thanks for helping.
  • d

    datajoely

    11/03/2021, 2:24 PM
    No worries @User
Powered by Linen
Title
d

datajoely

11/03/2021, 2:24 PM
No worries @User
View count: 1