Как исправить: ошибка типа: ожидаемая строка или байтовый объект
17 авг. 2022 г.
читать 1 мин
Одна ошибка, с которой вы можете столкнуться при использовании Python:
TypeError : expected string or bytes-like object
Эта ошибка обычно возникает, когда вы пытаетесь использовать функцию re.sub() для замены определенных шаблонов в объекте, но объект, с которым вы работаете, не состоит полностью из строк.
В следующем примере показано, как исправить эту ошибку на практике.
Как воспроизвести ошибку
Предположим, у нас есть следующий список значений:
#define list of values
x = [1, 'A', 2, 'B', 5, 'C', 'D', 'E']
Теперь предположим, что мы пытаемся заменить каждую небукву в списке пустой строкой:
import re
#attempt to replace each non-letter with empty string
x = re. sub('[^a-zA-Z]', '', x)
TypeError : expected string or bytes-like object
Мы получаем ошибку, потому что в списке есть определенные значения, которые не являются строками.
Как исправить ошибку
Самый простой способ исправить эту ошибку — преобразовать список в строковый объект, заключив его в оператор str() :
import re
#replace each non-letter with empty string
x = re. sub('[^a-zA-Z]', '', str (x))
#display results
print(x)
ABCDE
Обратите внимание, что мы не получили сообщение об ошибке, потому что использовали функцию str() для первого преобразования списка в строковый объект.
Результатом является исходный список, в котором каждая небуква заменена пробелом.
Примечание.Полную документацию по функции re.sub() можно найти здесь .
Дополнительные ресурсы
В следующих руководствах объясняется, как исправить другие распространенные ошибки в Python:
Как исправить KeyError в Pandas
Как исправить: ValueError: невозможно преобразовать число с плавающей запятой NaN в целое число
Как исправить: ValueError: операнды не могли транслироваться вместе с фигурами
I have read multiple posts regarding this error, but I still can’t figure it out. When I try to loop through my function:
def fix_Plan(location):
letters_only = re.sub("[^a-zA-Z]", # Search for all non-letters
" ", # Replace all non-letters with spaces
location) # Column and row to search
words = letters_only.lower().split()
stops = set(stopwords.words("english"))
meaningful_words = [w for w in words if not w in stops]
return (" ".join(meaningful_words))
col_Plan = fix_Plan(train["Plan"][0])
num_responses = train["Plan"].size
clean_Plan_responses = []
for i in range(0,num_responses):
clean_Plan_responses.append(fix_Plan(train["Plan"][i]))
Here is the error:
Traceback (most recent call last):
File "C:/Users/xxxxx/PycharmProjects/tronc/tronc2.py", line 48, in <module>
clean_Plan_responses.append(fix_Plan(train["Plan"][i]))
File "C:/Users/xxxxx/PycharmProjects/tronc/tronc2.py", line 22, in fix_Plan
location) # Column and row to search
File "C:UsersxxxxxAppDataLocalProgramsPythonPython36libre.py", line 191, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object
One error you may encounter when using Python is:
TypeError: expected string or bytes-like object
This error typically occurs when you attempt to use the re.sub() function to replace certain patterns in an object but the object you’re working with is not composed entirely of strings.
The following example shows how to fix this error in practice.
How to Reproduce the Error
Suppose we have the following list of values:
#define list of values
x = [1, 'A', 2, 'B', 5, 'C', 'D', 'E']
Now suppose we attempt to replace each non-letter in the list with an empty string:
import re
#attempt to replace each non-letter with empty string
x = re.sub('[^a-zA-Z]', '', x)
TypeError: expected string or bytes-like object
We receive an error because there are certain values in the list that are not strings.
How to Fix the Error
The easiest way to fix this error is to convert the list to a string object by wrapping it in the str() operator:
import re
#replace each non-letter with empty string
x = re.sub('[^a-zA-Z]', '', str(x))
#display results
print(x)
ABCDE
Notice that we don’t receive an error because we used str() to first convert the list to a string object.
The result is the original list with each non-letter replaced with a blank.
Note: You can find the complete documentation for the re.sub() function here.
Additional Resources
The following tutorials explain how to fix other common errors in Python:
How to Fix KeyError in Pandas
How to Fix: ValueError: cannot convert float NaN to integer
How to Fix: ValueError: operands could not be broadcast together with shapes
You might have used various functions in Python. While working with functions, there may be an error called “TypeError expected string or bytes-like object”. This is usually encountered when a function that you are using or have defined is fed an integer or float. It might be expecting a string or byte like object but as it has received something else, it raises an error.
The way to fix this error is to pass the correct argument to the function. You can change the syntax or convert the parameters into the required types.
We will take a closer look at the different scenarios where the error is raised. Subsequently, we will try to find their solutions.
Examples of TypeError expected string or bytes-like object
Example:
import re
# Declared Variable as Integer
strtoreplace = 1121
textonly = re.sub("[^a-zA-Z]", " ",strtoreplace)
print('Print Value: ', textonly)
Output:
Traceback (most recent call last):
File "pyprogram.py", line 6, in <module>
textonly = re.sub("[^a-zA-Z]", " ",strtoreplace)
File "C:Python38libre.py", line 208, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object

Solution Example:
import re
# Declared Variable as Integer
strtoreplace = 1121
textonly = re.sub("[^a-zA-Z]", " ",str(strtoreplace))
print('Print Value: ', textonly)
This error is encountered as there a couple of values in the code that are floats. In order to run the code successfully, you have to convert some value into strings. Before passing the values into the re.sub() function, you can convert into a string using the str() function.
Whenever we are working with our code to develop something and we write some code, then it may be possible our code encounters any type of error. One of those error is Typeerror, which indicates that we have not given the type of data required. For example, if we try to add 3 + “five”, then we will encounter an error named typeerror because the summation between an integer and a string is not supported in python.
Whenever this type of error occurs, we have to make sure that we are passing the same type of data that is required to complete the operation.
So, in the above sum we are required to pass an integer to add in 3.
It will eliminate our error and the operation will be successfully performed.
In this article we are going to see what causes the error named typeerror: expected string or bytes like object. Basically, we will encounter this type of error, when we are working with a regular expression and we are trying to replace any value with some other value.
Example 1:
|
import pandas as pd import re data = pd.DataFrame({‘Name’:[«Alpha», «Beta», 12, «Gamma»]}) print(data) data[‘Name’] = [re.sub(str(i),«Omega», i) if i == 12 else i for i in data[‘Name’]] print(data) |
Output:
|
PS C: Users ASUS Desktop Crazy Programmer Work > python —u » c : Users ASUS Desktop Crazy Programmer Work test.py « Name 0 Alpha 1 Beta 2 12 3 Gamma Traceback (most recent call last): File «c : Users ASUS Desktop Crazy Programmer Work test.py «, line 8, in <module> data[‘Name’] = [re.sub(str(i),«Omega», i) if i == 12 else i for i in data[‘Name’]] File «c : Users ASUS Desktop Crazy Programmer Work test.py «, line 8, in <listcomp> data[‘Name’] = [re.sub(str(i),«Omega», i) if i == 12 else i for i in data[‘Name’]] File «C : Program Files Python382 lib re.py», line 208, in sub return _compile(pattern, flags).sub(repl, string, count) TypeError: expected string or bytes—like object |
Here we have a DataFrame in which there are some data. In the initial stage when we are trying to print it, it is printed successfully.
Let us suppose we want to change the value of 12 in the Name column.
When we are trying to change it, it is giving the same error as we have discussed earlier.
So we have to check our code that whether the bug is there. As I described earlier this error occurs when we are trying to give any other datatype value different from its normal type which is required in the operation.
Here we are using a regular expression to replace the integer value with a string value.
So, we have to make a simple change in our code to execute it successfully.
As we can see from the documentation of regular expression that re.sub() function required its third parameter as a string but in the code we have passes it as integers.
Solution:
To resolve it, we have to pass the third parameter named I as a string. We have to write str(i) instead of I to remove this error.
|
import pandas as pd import re data = pd.DataFrame({‘Name’:[«Alpha», «Beta», 12, «Gamma»]}) print(data) data[‘Name’] = [re.sub(str(i),«Omega», str(i)) if i == 12 else i for i in data[‘Name’]] print(data) |
Output:
|
PS C: Users ASUS Desktop Crazy Programmer Work > python —u » c : Users ASUS Desktop Crazy Programmer Work test.py « Name 0 Alpha 1 Beta 2 12 3 Gamma Name 0 Alpha 1 Beta 2 Omega 3 Gamma |
As we can see that after passing the third parameter as string, we are successfully able to perform the desired operation using regular expression.
We have successfully changed the value of integer 12 to string Omega.
Example 2:
|
import re list1 = [‘Alpha’, ‘Beta’, ‘Gamma’, 40, 30] list1 = [re.sub(str(item), «this», item) if item in range(10, 41) else item for item in list1 ] print(list1) |
Output:
|
PS C: Users ASUS Desktop Crazy Programmer Work > python —u » c : Users ASUS Desktop Crazy Programmer Work test.py « Traceback (most recent call last): File «c : Users ASUS Desktop Crazy Programmer Work test.py «, line 59, in <module> list1 = [re.sub(str(item), «this», item) if item in range(10, 41) else item for item in list1 ] File «c : Users ASUS Desktop Crazy Programmer Work test.py «, line 59, in <listcomp> list1 = [re.sub(str(item), «this», item) if item in range(10, 41) else item for item in list1 ] File «C: Program Files Python382 lib re.py «, line 208, in sub return _compile(pattern, flags).sub(repl, string, count) TypeError: expected string or bytes—like object |
In this example, we are trying to replace some values in the list which are integers ranging from 10 to 40. We are performing this operation using sub() function in a regular expression.
As we can see that we have encountered with the same error. Whenever we are working with sub() function, then we have to cross check that we are passing the correct argument type else we will encounter the same error and it is a little bit difficult to find where the exact error is.
As we can see that we are passing the third argument as an integer that why we have encountered the error.
Solution:
We can resolve this error by passing the third argument as string type by converting it into a string.
Let’s see the corrected code and its output.
|
import re list1 = [‘Alpha’, ‘Beta’, ‘Gamma’, 40, 30] list1 = [re.sub(str(item), «this», str(item)) if item in range(10, 41) else item for item in list1 ] print(list1) |
Output:
|
PS C: Users ASUS Desktop Crazy Programmer Work > python —u » c : Users ASUS Desktop Crazy Programmer Work test.py « [‘Alpha’, ‘Beta’, ‘Gamma’, ‘this’, ‘this’] |
As we can see that after passing the third argument as a string, we have successfully fixed the error which was shown earlier.
Conclusion
This type of error mostly occurs when we are working with sub() function in a regular expression. This work of this function is to replace a given pattern with some string. We have to be careful when we are passing its arguments. We have to check which type of argument is required to perform the operation. Whenever we pass the argument of other type which is not required then we will encounter this type of error.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.
Already on GitHub?
Sign in
to your account
Labels
kind/bug
Something isn’t working as expected
Comments
- I am on the latest Poetry version.
- I have searched the issues of this repo and believe that this is not a duplicate.
- If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption).
- OS version and name: based on official docker image
python:3.8-slim-buster(see below for Dockerfile) - Poetry version: 1.1.4
- virtualenv version: 20.4.2
- Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/wichert/ba05b0505c24ee7c71c5d7deb2ac2cfe
Issue
When I run poetry install in a private GitHub runner on an GHES installation I started getting an error (see further below for full output):
TypeError
expected string or bytes-like object
at /usr/local/lib/python3.8/site-packages/poetry/core/utils/helpers.py:24 in canonicalize_name
Debug output
Creating virtualenv amp-portal in /__w/backend/backend/amp-portal/.venv
Using virtualenv: /__w/backend/backend/amp-portal/.venv
Stack trace:
11 /usr/local/lib/python3.8/site-packages/clikit/console_application.py:131 in run
129│ parsed_args = resolved_command.args
130│
→ 131│ status_code = command.handle(parsed_args, io)
132│ except KeyboardInterrupt:
133│ status_code = 1
10 /usr/local/lib/python3.8/site-packages/clikit/api/command/command.py:120 in handle
118│ def handle(self, args, io): # type: (Args, IO) -> int
119│ try:
→ 120│ status_code = self._do_handle(args, io)
121│ except KeyboardInterrupt:
122│ if io.is_debug():
9 /usr/local/lib/python3.8/site-packages/clikit/api/command/command.py:163 in _do_handle
161│ if self._dispatcher and self._dispatcher.has_listeners(PRE_HANDLE):
162│ event = PreHandleEvent(args, io, self)
→ 163│ self._dispatcher.dispatch(PRE_HANDLE, event)
164│
165│ if event.is_handled():
8 /usr/local/lib/python3.8/site-packages/clikit/api/event/event_dispatcher.py:22 in dispatch
20│
21│ if listeners:
→ 22│ self._do_dispatch(listeners, event_name, event)
23│
24│ return event
7 /usr/local/lib/python3.8/site-packages/clikit/api/event/event_dispatcher.py:89 in _do_dispatch
87│ break
88│
→ 89│ listener(event, event_name, self)
90│
91│ def _sort_listeners(self, event_name): # type: (str) -> None
6 /usr/local/lib/python3.8/site-packages/poetry/console/config/application_config.py:141 in set_installer
139│
140│ poetry = command.poetry
→ 141│ installer = Installer(
142│ event.io,
143│ command.env,
5 /usr/local/lib/python3.8/site-packages/poetry/installation/installer.py:65 in __init__
63│ self._installer = self._get_installer()
64│ if installed is None:
→ 65│ installed = self._get_installed()
66│
67│ self._installed_repository = installed
4 /usr/local/lib/python3.8/site-packages/poetry/installation/installer.py:561 in _get_installed
559│
560│ def _get_installed(self): # type: () -> InstalledRepository
→ 561│ return InstalledRepository.load(self._env)
562│
3 /usr/local/lib/python3.8/site-packages/poetry/repositories/installed_repository.py:118 in load
116│ path = Path(str(distribution._path))
117│ version = distribution.metadata["version"]
→ 118│ package = Package(name, version, version)
119│ package.description = distribution.metadata.get("summary", "")
120│
2 /usr/local/lib/python3.8/site-packages/poetry/core/packages/package.py:51 in __init__
49│ Creates a new in memory package.
50│ """
→ 51│ super(Package, self).__init__(
52│ name,
53│ source_type=source_type,
1 /usr/local/lib/python3.8/site-packages/poetry/core/packages/specification.py:19 in __init__
17│ ): # type: (str, Optional[str], Optional[str], Optional[str], Optional[str], Optional[List[str]]) -> None
18│ self._pretty_name = name
→ 19│ self._name = canonicalize_name(name)
20│ self._source_type = source_type
21│ self._source_url = source_url
TypeError
expected string or bytes-like object
at /usr/local/lib/python3.8/site-packages/poetry/core/utils/helpers.py:24 in canonicalize_name
20│ _canonicalize_regex = re.compile("[-_]+")
21│
22│
23│ def canonicalize_name(name): # type: (str) -> str
→ 24│ return _canonicalize_regex.sub("-", name).lower()
25│
26│
27│ def module_name(name): # type: (str) -> str
28│ return canonicalize_name(name).replace(".", "_").replace("-", "_")
Dockerfile for base docker image
ARG DEBIAN_VERSION=buster
ARG PYTHON_VERSION=3.8
FROM python:${PYTHON_VERSION}-slim-${DEBIAN_VERSION}
ENV
DEBIAN_FRONTEND=noninteractive
PYTHONUNBUFFERED=1
PYTHONDONTWRITEBYTECODE=1
POETRY_VIRTUALENVS_IN_PROJECT=true
POETRY_NO_INTERACTION=1
RUN set -x
&& apt-get update
&& apt-get install -yq --no-install-recommends libpq-dev build-essential sudo
&& rm -rf /var/lib/apt/lists/*
RUN pip install poetry==1.1.4
We should try to narrow it down, maybe there is a dependency somewhere with packaging issues.
Some extra information:
- we use a GHES installation with multiple runners. On some runners it fails every time, on others it always succeeds.
- this is not dependent on the local
.venvdirectory: that is newly created on every run - it is not dependent on what is in the users home directory: that is part of the docker container and is identical on all runners.
So there is something very specific to a runner that is independent from the docker image that triggers this error.
I see. This is good info. This changes my perspective on this issue.
Perhaps this could be a broken cache entry somewhere?
I’m adding @netcaptors here; he is a member of the team that manages the github runners and has more insight into their configuration than I have.
I’m out of my depth on this. I’ll ping @stephsamson and @finswimmer maybe they’ll be able to give more insight or redirect to other maintainers.
But you seem to say it’s rather new behaviour, although there was no new poetry release in a while, right?
This particularly error is new for us today as far as I can tell. I expect that there is something in the environment that poetry needs to deal with that changed and triggered this error.
Doubtful, we had not done a rebuild of our docker image at the moment this error started occurring as far as I know.
As magically as this error started appearing, it seems to have disappeared today. As far as I know the only change that happened is that chmod -R github-runner:github-runner /home/github-runner/runner/_work/ was run on the runner instance that was showing the error. The best theory I have is that there was a directory or file somewhere that was not readable by the user, which triggered this error. Perhaps a .dist-info directory that was not readable?
I saw this issue locally today on my mac. I used poetry 1.1.4 and it worked for me until today, for some reason, poetry install doesn’t work and reported the same error in this issue. Commands like poetry show stops working too with the same error.
I tried revert everything to a specific git commit I confirmed working recently (delete the entire folder and fetch from git again, so it doesn’t seem to me a permission issue), but the same error was reported.
I got the same problem. I dropped into PDB and tried to track it down.
name is None and it comes from here:
https://github.com/python-poetry/poetry/blob/1.1.4/poetry/repositories/installed_repository.py#L115
Then if I run this in PDB:
pp [x.metadata["Name"] for x in metadata.distributions(path=[env.sys_path[4]])]
I get for my specific .venv/project I get something like:
[ '...',
'dbus-python',
None,
'pluggy',
'flake8',
'...']
Note the None.
Now let’s find the culprit:
pp [x.files if x.metadata["name"] is None else "SKIP" for x in metadata.distributions(path=[env.sys_path[4]])]
[ '...',
'SKIP',
[PackagePath('more_itertools-8.7.0.dist-info/INSTALLER'),
PackagePath('more_itertools-8.7.0.dist-info/LICENSE'),
PackagePath('more_itertools-8.7.0.dist-info/METADATA'),
PackagePath('more_itertools-8.7.0.dist-info/RECORD'),
PackagePath('more_itertools-8.7.0.dist-info/REQUESTED'),
PackagePath('more_itertools-8.7.0.dist-info/WHEEL'),
PackagePath('more_itertools-8.7.0.dist-info/direct_url.json'),
PackagePath('more_itertools-8.7.0.dist-info/top_level.txt'),
PackagePath('more_itertools/__init__.py'),
PackagePath('more_itertools/__init__.pyi'),
PackagePath('more_itertools/__pycache__/__init__.cpython-39.pyc'),
PackagePath('more_itertools/__pycache__/more.cpython-39.pyc'),
PackagePath('more_itertools/__pycache__/recipes.cpython-39.pyc'),
PackagePath('more_itertools/more.py'),
PackagePath('more_itertools/more.pyi'),
PackagePath('more_itertools/py.typed'),
PackagePath('more_itertools/recipes.py'),
PackagePath('more_itertools/recipes.pyi')],
'SKIP',
'...']
For some reason PathDistribution object representing more_itertools package has empty metadata and version fields.
Interestingly enough it’s also the only dependency of my project dependencies which upgraded when I last ran poetry update. Not sure if that’s related, or they broke something.
Hope this helps.
EDIT:
I actually checked the METADATA file in more_itertools-8.7.0.dist-info directory and it’s empty.
Extra info:
pip version in venv: 21.0.1
pip version outside: 20.3.3
poetry version: 1.1.4
EDIT 2:
I bet it has something to do with pip version as mentioned before.
EDIT 3:
I ended up having to clear ~/.cache/pypoetry seems like the whole artifact was corrupted. Without that even deleting and reinstalling .venv would crash on poetry install.
I was also having the same problem.
In my case, I ran poetry shell to test out running pyInstaller command inside the shell. Once I exited the shell and tried to run command poetry run or poetry install, I got expected string or bytes-like object error message.
MaikuMori’s third solution, removing ~/.cache/pypoetry did the trick and solved the issue.
Update:
Removing ~/.cache/pypoetry did work temporarily. The first command either poetry install or poetry run will work, but on subsequent command still got the same error message. To get around it, remove the cache then run poetry command again and keeps repeating the cycle.
lazToum
added a commit
to lazToum/poetry
that referenced
this issue
Mar 21, 2021
We also get those expected string or bytes-like object a lot, but we are working in a conda environment on MacOS. For us, it seems to happen more often when we are changing branch in our soft repo, then doing a poetry install, or when updating a dependency to use the path directive ('toto' = {path='<some-path>', develop=true}), for version. Unfortunately, I didn’t find a way to consistently reproduce those errors.
First, I start with a pyenv-installed python-3.8.8:
$ pyenv install 3.8.8
$ pyenv global 3.8.8
I try to reproduce the problem after a fresh installation of poetry-1.1.5:
$ mkdir debug
$ cd debug
$ poetry new somepackage
$ cd somepackage
$ poetry install # This creates the virtual environment in ~/.cache/pypoetry/virtualenvs
TypeError
expected string or bytes-like object
at ~/.poetry/lib/poetry/_vendor/py3.8/poetry/core/utils/helpers.py:27 in canonicalize_name
23│ _canonicalize_regex = re.compile(r"[-_]+")
24│
25│
26│ def canonicalize_name(name): # type: (str) -> str
→ 27│ return _canonicalize_regex.sub("-", name).lower()
28│
29│
30│ def module_name(name): # type: (str) -> str
31│ return canonicalize_name(name).replace(".", "_").replace("-", "_")
Now, I inspect the newly-created virtual environment’s site-packages
$ cd ~/.cache/pypoetry/virtualenvs/somepackage-RAiBNyBV-py3.8/lib/python3.8/site-packages
$ ls -al
total 44K
drwxr-xr-x 2 myuser mygrp 4.0K Apr 7 09:23 _distutils_hack
drwxr-xr-x 4 myuser mygrp 4.0K Apr 7 09:23 pip
drwxr-xr-x 5 myuser mygrp 4.0K Apr 7 09:23 pkg_resources
drwxr-xr-x 2 myuser mygrp 4.0K Apr 7 09:23 __pycache__
drwxr-xr-x 6 myuser mygrp 4.0K Apr 7 09:23 setuptools
drwxr-xr-x 4 myuser mygrp 4.0K Apr 7 09:23 wheel
drwxr-xr-x 2 myuser mygrp 4.0K Apr 7 09:23 pip-21.0.1.dist-info
drwxr-xr-x 2 myuser mygrp 4.0K Apr 7 09:23 setuptools-52.0.0.dist-info
drwxr-xr-x 2 myuser mygrp 4.0K Apr 7 09:23 wheel-0.36.2.dist-info
-rw-r--r-- 1 myuser mygrp 0 Apr 7 09:23 distutils-precedence.pth
-rw-r--r-- 1 myuser mygrp 18 Apr 7 09:23 _virtualenv.pth
-rw-r--r-- 1 myuser mygrp 5.6K Apr 7 09:23 _virtualenv.py
-rw-r--r-- 1 myuser mygrp 0 Apr 7 09:23 pip-21.0.1.virtualenv
-rw-r--r-- 1 myuser mygrp 0 Apr 7 09:23 setuptools-52.0.0.virtualenv
-rw-r--r-- 1 myuser mygrp 0 Apr 7 09:23 wheel-0.36.2.virtualenv
They seem to be empty, even the pip folder has zero-length files:
$ ls -al pip
total 8.0K
drwxr-xr-x 13 myuser mygrp 4.0K Apr 7 09:23 _internal
drwxr-xr-x 19 myuser mygrp 4.0K Apr 7 09:23 _vendor
-rw-r--r-- 1 myuser mygrp 0 Apr 7 09:23 __init__.py
-rw-r--r-- 1 myuser mygrp 0 Apr 7 09:23 __main__.py
Is this intentional?
I did all of this using previous poetry versions (1.1.0 and 1.1.4 if I remember correctly). I see the same behavior.
I see empty site-packages if I use:
- pyenv installed python
- python that I compiled myself
site-packages are populated if I use the system python.
The rubber ducky problem solving method has been demonstrated. Writing down my problem in #3628 (comment) has helped a lot in framing it.
This is what I did to get around the problem of empty site-packages
# system python is only 3.7. I need 3.8.
$ pyenv install 3.8.8
$ pyenv versions
* system (set by ~/.pyenv/version)
3.8.8
# Use system python to do all the poetry initialization
$ pyenv global system
$ python --version
Python 3.7.3
$ mkdir debug
$ cd debug
$ poetry new somepackage
$ cd somepackage
$ poetry env use ~/.pyenv/versions/3.8.8/python
$ poetry install
$ poetry shell
$ python --version
Python 3.8.8
That should do it.
I’ve stumbled into the same issue today, seemingly out of the blue. This morning I did a system upgrade (pacman -Syu) and after the first poetry install the message started appearing.
I had used pyenv to use Python 3.7.8. I tried clearing the cache and recreating the venv, but no luck. I then tried the system Python to create the venv (Python 3.9.2), but same thing.
I could run poetry update repeatedly in a clean environment, but after the first run of poetry install the message appears every time (even on poetry update).
I then tried installing another project, which worked fine. Then I tried installing the old project (https://github.com/ActivityWatch/aw-client) again, which gave the output:
Installing dependencies from lock file
Package operations: 9 installs, 0 updates, 0 removals
• Installing lazy-object-proxy (1.6.0)
• Installing wrapt (1.12.1)
• Installing astroid (2.5.3)
• Installing coverage (5.5)
• Installing isort (5.8.0)
• Installing mccabe (0.6.1)
• Installing pylint (2.7.4)
• Installing pytest-cov (2.11.1)
• Installing tabulate (0.8.9)
Installing the current project: aw-client (0.3.1)
I assume one of these dependencies are to blame, but I don’t know which (and yes, they are all up to date).
Edit: Weirdly enough the same issue doesn’t seem to occur in CI (ActivityWatch/aw-client#56), so perhaps one of the dependencies aren’t to blame after all? Or it’s just that poetry install or similar is never ran twice…
Edit: The issue ended up being the .egg-link directories that were in various places in my project. I deleted them, and now no issues!
Found them all with: find . -maxdepth 2 | grep .egg-info
I’m not sure if this is useful, but I was having this issue when deploying via Docker to Digital Ocean App Platform. I changed my Docker image base as such:
from:
FROM python:3.8.1-slim-buster
to:
In my case poetry fails with a similar error when I’m trying to add a package:
poetry add "git+https://github.com/streamlit/streamlit.git#0.81.2.dev20210506"
I am not sure if it has anything to do with what you guys are experiencing but after get this problem localy in serveral projects and find this issue I started to debug poetry and found out that the issue was happening because, for some reason, I had a .egg-info directory. I could than see that poetry was checking on it for installed packages and there was a distribution object with no name in its metadata.
| name = canonicalize_name(distribution.metadata[«name»]) |
Reproducing the error: (I dont know how it would behave with a valid .egg-info directory)
# Working as expected
~/open-offertator$ poetry install
Installing dependencies from lock file
No dependencies to install or update
Installing the current project: open_offertator (0.1.0)
# Create a .egg-info directory
~/open-offertator$ mkdir open_offertator.egg-info/
# Try poetry again
~/open-offertator$ poetry install
TypeError
expected string or bytes-like object
at ~/.pyenv/versions/3.9.4/lib/python3.9/site-packages/poetry/core/utils/helpers.py:27 in canonicalize_name
23│ _canonicalize_regex = re.compile(r"[-_]+")
24│
25│
26│ def canonicalize_name(name): # type: (str) -> str
→ 27│ return _canonicalize_regex.sub("-", name).lower()
28│
29│
30│ def module_name(name): # type: (str) -> str
31│ return canonicalize_name(name).replace(".", "_").replace("-", "_")
I hope it helps.
I have a minimal reproduction pyproject.toml file that may be the same issue. I was trying to add a spacy model directly to my deps (the en-core-web-sm line below), but I ultimately had to specify it as a URL dep to get poetry.lock to generate.
[tool.poetry]
name = "failure-to-install"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]
[tool.poetry.dependencies]
python = "^3.9"
en-core-web-sm = { git = "https://github.com/explosion/spacy-models", tag = "en_core_web_sm-3.0.0" }
[tool.poetry.dev-dependencies]
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
Hope this helps!
This has nothing to do with anaconda, I’ve had this without using anaconda.
#3628 (comment)
For some of us, it is directly due to anaconda.
I am currently facing this without anaconda (actually deactivated an anaconda env beacuse of some comments above) on pure venv. Still no recourse.
poetry install works fine the first time,
but the second time around, fails with this errorm which is really bizarre.
Did you try reinstalling poetry, making sure your anaconda environment wasn’t active during install? I’d use the same python binary whenever you run poetry commands.
For some of us, it is directly due to anaconda.
It looks like it triggers the bug, but there are other triggers too.
For some reason, the download is corrupted, which then corrupts the cache. Without clearing the cache, it will continue to install the corrupted version.
And by corrupted I mean the METADATA file being empty for the package.
For me it was related to the .venv folder created by poetry:
👉 Deleting that folder and running poetry update worked for me 😄 .
I’m using pyenv and poetry with the pyenv-virtualenv extension.
Along the lines of @Matesanz ‘s solution,
pyenv virtualenv-delete <offending-venv>
pyenv virtualenv <python-version> <offending-venv>
poetry install
worked for me.
jeff-hykin
added a commit
to jeff-hykin/poetry
that referenced
this issue
Mar 4, 2022
What worked for me:
poetry env list --full-path which returns path
rm -rf path
poetry update
For me, the issue was triggered by installing from a git URL, which worked initially and then became corrupted. I was only using poetry, not conda and only using pip implicitly.
Even after deleting the .venv, poetry install or poetry update continues to give me this
TypeError expected string or bytes-like object
Try deleting the poetry.lock also before the poetry install and update
I’m not sure why you’d use poetry with anaconda? Poetry is an environment manager for virtual envs, why would you make a conda environment too? In my case, I had previously used anaconda environments and use them for some old projects still, while now having migrated solely to poetry. I just needed to switch out of that old anaconda env when switching to a separate poetry managed project.
Because we have as deps some software available only in conda and not pypi (and they won’t be), but we can still use a big part without the need for a conda environment.
Plus, were I work, it is used to handle all aspect of the development (like publishing the package to a private pypi, handling the dependencies, etc), not just the environment (on which it can causes us some trouble when we have to work with conda).
@aalok-sathe You are running poetry inside a conda environment ? If yes, you can try to run the following snippet:
from os import environ, remove, rmdir from shutil import rmtree from sys import version_info import importlib_metadata as metadata MODULES_PATH = [ f"{environ['CONDA_PREFIX']}/lib/python{version_info.major}.{version_info.minor}/site-packages", f"{environ['HOME']}/.local/lib/python{version_info.major}.{version_info.minor}/site-packages", ] if __name__ == '__main__': for pkg in filter(lambda a: a.metadata["name"] is None, metadata.distributions(path=MODULES_PATH)): print("Erasing", pkg._path) if pkg._path.is_dir(): rmtree(pkg._path) else: remove(pkg._path)
(I’ve already posted it, but can’t recall in which issue).
This was referenced
May 19, 2022
The TypeError «expected string or bytes-like object» appears to obscure various kinds of issues people posted here. It’d be great if the actual underlying root causes can be surfaced instead.
In my case, I ran into the issue in the following combination:
- create and use a conda env
- use
poetry installto install package A from source. - use
poetry installto install package B from source.
TypeError is encountered in installing package B.
I later realize doing poetry install on multiple packages in a single conda environment is probably problematic, if not outright unsupported. If that’s indeed the case, it’d be great if a more explicit error is raised instead.
abn
mentioned this issue
Jun 6, 2022
Still getting this error on Poetry 1.1.13. I have no .egg-info folders anywhere. Deleting my virtual environment is not an option. I have several stubs there that need to stay in place and re-installing my large project every single time this happens is not reasonable.
Still getting this error on Poetry 1.1.13. I have no
.egg-infofolders anywhere. Deleting my virtual environment is not an option. I have several stubs there that need to stay in place and re-installing my large project every single time this happens is not reasonable.
The fix is in Poetry 1.2b2 — Poetry 1.1.x is not seeing releases as we work up to a full release of 1.2.
Still getting this error on Poetry 1.1.13. I have no
.egg-infofolders anywhere. Deleting my virtual environment is not an option. I have several stubs there that need to stay in place and re-installing my large project every single time this happens is not reasonable.The fix is in Poetry 1.2b2 — Poetry 1.1.x is not seeing releases as we work up to a full release of 1.2.
Yup. Discovered that after commenting about your poor documentation.
For me, version 1.2.0b2 found multiple kiwisolver .dist-info folders in my site-packages. One for an older version and one for the latest. I had no .egg-info folders anywhere, so I think this was the culprit.
Still getting this error on Poetry 1.1.13. I have no
.egg-infofolders anywhere. Deleting my virtual environment is not an option. I have several stubs there that need to stay in place and re-installing my large project every single time this happens is not reasonable.The fix is in Poetry 1.2b2 — Poetry 1.1.x is not seeing releases as we work up to a full release of 1.2.
Yup. Discovered that after commenting about your poor documentation.
Poetry is free software developed by volunteers without any expectation of warranty. I am sorry that you had to struggle to find the relevant documentation, and I will make a note of improving it before release. Note that we are still in beta for 1.2 and things are therefore still in flux. Please refrain from using an unnecessarily hostile or confrontational tone.
For me, version 1.2.0b2 found multiple
kiwisolver.dist-infofolders in my site-packages. One for an older version and one for the latest. I had no.egg-infofolders anywhere, so I think this was the culprit.
I’m glad you were able to sort the issues out. Please test 1.2 in your environment and open issues for any regressions or deficiencies (like documentation) you may find.
Apologies, my frustration came out and that was unprofessional. I appreciate your help. Thank you for responding so quickly.
I tried using 1.2.0rc1 and it fails a different way so I can’t tell if it fixes the name problem. It’s in a similar area of code and has the same kind of feel, so I figured I’d at least mention it here since I really don’t have time to open a whole new ticket:
• Updating certifi (2021.10.8 /Users/kentpitman/Library/Caches/pypoetry/artifacts/61/99/d0/9b77a70db9834d3d8281903159311224b92e6ac1cdab98dca7f6367740/certifi-2021.10.8-py2.py3-none-any.whl -> 2022.6.15): Failed
AssertionError
at fsenv/lib/python3.7/site-packages/poetry/utils/env.py:343 in find_distribution_files_with_name
339│ ) -> Iterable[Path]:
340│ for distribution in self.distributions(
341│ name=distribution_name, writable_only=writable_only
342│ ):
→ 343│ assert distribution.files is not None
344│ for file in distribution.files:
345│ if file.name == name:
346│ yield Path(
347│ distribution.locate_file(file), # type: ignore[no-untyped-call]
That assertion does behave helpfully because it stops everything dead in its tracks. Is it really that fatal? I would think it would be less disruptive to merge lines 343 and 344 as
for file in distribution.files or []: # sometimes distribution.files is None
possibly adding a log.warning if you want someone to know about the problem? Putting assert in production code seems iffy. I thought that was only for testing.
if you care then raise a new ticket and say how to reproduce, no point in making updates in unrelated and already-closed issues
Definitely do open a new issue if you can reproduce this consistently in a container/VM.
With regard to assert in production code, it’s there for typing purposes and is very helpful for catching this kind of bug. It should be impossible for a None to sneak in there according to the intention of the authors of that section of code, and yet it happened — this is why typing (and typing asserts) are very helpful for writing robust Python code.
I’ll try to open a ticket if I get time, but as you probably know, it takes time to do a good ticket that isn’t overly complicated. It helps to know that you think this is not a known problem, since that makes the investment in time seem more justified. Thanks.
Still experience this 🙁
This is a closed issue that could refer to a wild assortment of different root causes — if you are running into this, please narrow it down to a minimal reproduction and open an issue so we can try to understand how to solve it.
python-poetry
locked as off-topic and limited conversation to collaborators
Sep 2, 2022
Labels
kind/bug
Something isn’t working as expected
Приветствую, ситуация следующая. Имеется проект блога на flask, подключил админку по средствам flask-admin. Настроил через app.py путем добавления:
admin.add_view(ModelView(Post, db.session))
admin.add_view(ModelView(Tag, db.session))
В первом случае работа с постами идет без нареканий (создать пост, редактировать, удалить). Всё отрабатывает на ура.
Но что касаться Tag то вот тут и возникает проблема. Редактируються уже созданные (в ручную) теги замечательно, но вот при создании тега, после заполнения полей вылетает следующие сообщение:
builtins.TypeError
TypeError: expected string or bytes-like object
Traceback (most recent call last):
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask/app.py", line 1997, in __call__
return self.wsgi_app(environ, start_response)
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask/app.py", line 1985, in wsgi_app
response = self.handle_exception(e)
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask/app.py", line 1540, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
raise value
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
raise value
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask_admin/base.py", line 69, in inner
return self._run_view(f, *args, **kwargs)
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask_admin/base.py", line 368, in _run_view
return fn(self, *args, **kwargs)
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask_admin/model/base.py", line 1997, in create_view
model = self.create_model(form)
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask_admin/contrib/sqla/view.py", line 1079, in create_model
if not self.handle_view_exception(ex):
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask_admin/contrib/sqla/view.py", line 1062, in handle_view_exception
return super(ModelView, self).handle_view_exception(exc)
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/flask_admin/contrib/sqla/view.py", line 1073, in create_model
model = self.model()
File "<string>", line 4, in __init__
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/sqlalchemy/orm/state.py", line 417, in _initialize_instance
manager.dispatch.init_failure(self, args, kwargs)
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
raise value
File "/home/moonz/Рабочий стол/Flask/venv/lib/python3.6/site-packages/sqlalchemy/orm/state.py", line 414, in _initialize_instance
return manager.original_init(*mixed[1:], **kwargs)
File "/home/moonz/Рабочий стол/Flask/models.py", line 50, in __init__
self.slug = slugify(self.name)
File "/home/moonz/Рабочий стол/Flask/models.py", line 8, in slugify
return re.sub(pattern, '-', s) # Заменяет их на дефис
File "/usr/lib/python3.6/re.py", line 191, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object
Как я понимаю, на вход поступает неверный тип данных, но вот где именно проблема (а точнее в Чём) понять не могу. Буду благодарен любым подсказкам, заранее благодарю.
Уведомления
- Начало
- » Python для новичков
- » Множественные замены в текстовом файле
#1 Авг. 25, 2017 11:55:14
Множественные замены в текстовом файле
Добрый день.
Подскажите пожалуйста как в текстовом файле сделать замену по шаблону, с сохранением строк и последующей записью.
Например. Нужно заменить во всем файле “age” на “years” и “fname” на “surname”
pname 1; fname 1; age 1 pname 2; fname 2; age 2 pname 3; fname 3; age 3
Офлайн
- Пожаловаться
#2 Авг. 25, 2017 12:10:00
Множественные замены в текстовом файле
Метод replace
С дураками и сектантами не спорю, истину не ищу.
Ели кому-то правда не нравится, то заранее извиняюсь.
Офлайн
- Пожаловаться
#3 Авг. 25, 2017 12:24:43
Множественные замены в текстовом файле
Почему выдает ошибку?
File “C:Python 3.6libre.py”, line 191, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object
def parsing(): rpl = [['Нормальный ', ''], ['Дверь', ''], ['Дата: ', ''], [' Область: ', '']] f_in = open('in.csv', 'r') f_out = open('out.txt', 'w', encoding='utf-8') for string in rpl: input_file = re.sub(str(string[0]),str(string[1]),f_in) f_out.write(input_file) f_in.close() f_out.close()
Отредактировано Ubhra (Авг. 25, 2017 12:57:48)
Офлайн
- Пожаловаться
#4 Авг. 25, 2017 13:12:05
Множественные замены в текстовом файле
Ubhra
Почему выдает ошибку?
в re.sub третий параметр должен быть строкой или набором байт. а у тебя это объект файла.
Офлайн
- Пожаловаться
#5 Авг. 25, 2017 13:43:41
Множественные замены в текстовом файле
open(‘in.csv’, ‘r’)
должен ведь возвращать содержимое файла одной строкой
П.С. Аааа точно его же прочесть нужно:
input_file = re.sub(str(string[0]),str(string[1]),f_in.read())
Вот только он создает пустой файл.
Отредактировано Ubhra (Авг. 25, 2017 13:46:35)
Офлайн
- Пожаловаться
#6 Авг. 25, 2017 14:05:51
Множественные замены в текстовом файле
Вот так частично заработало, но изменяет только первую запись:
def parsing(): rpl = [['Нормальный ', ''],['Дверь', ''],['Дата: ', ''],[' Область: ', '']] f_in = open('in.csv', 'r') f_out = open('out.txt', 'w', encoding='utf-8') for string in rpl: f_out.write(re.sub(str(string[0]),str(string[1]),f_in.read())) f_in.close() f_out.close() print('ok')
Офлайн
- Пожаловаться
#7 Авг. 25, 2017 14:29:02
Множественные замены в текстовом файле
Питон зверюга всеядная. У меня так заработало 😀 :
def parsing(): f_in = open('in.csv', 'r') f_out = open('out.txt', 'w', encoding='utf-8') nf1 = re.sub(str('Нормальный вход по ключу'),str('вход'),f_in.read()) nf2 = re.sub(str('Нормальный выход по ключу'),str('выход'),nf1) nf3 = re.sub(str('Дверь'),str(''),nf2) nf4 = re.sub(str('Дата: '),str(''),nf3) nf5 = re.sub(str(' Область: '),str(''),nf4) nf6 = re.sub(str('"'),str(''),nf5) f_out.write(nf6) f_in.close() f_out.close() print('ok')
Офлайн
- Пожаловаться
#9 Апрель 25, 2020 21:17:13
Множественные замены в текстовом файле
Добрый день!
Не подскажете, почему выдаёт ту же самую “зловредную”
ошибку “TypeError: expected string or bytes-like object”?
import re, urllib.request
class HTMLFinder:
def findTag(website, tag):
url = urllib.request.urlopen(website)
tags = url.read()
tagsF = ‘’’»’
for line in tags:
if re.search(tag, line):
tagsF = tagsF + line + ‘/n’
if len(tagsF) < 3: return False
else: return tagsF
Ну я в принципе-то всё правильно написал, пробелы есть, а пишет всё то же(((
Отредактировано PyBeg (Апрель 25, 2020 21:19:47)
Офлайн
- Пожаловаться
#10 Апрель 26, 2020 04:21:49
Множественные замены в текстовом файле
PyBeg
Не подскажете, почему выдаёт ту же самую “зловредную” ошибку “TypeError: expected string or bytes-like object”?
Потому что принятые данные в байтах надо раскодировать через метод .decode() в юникод-строку. Для раскодирования надо использовать кодировку, которая передаётся вместе с данными. Можно и наугад раскодировать, использовав распространённую кодировку utf-8. Для одного случая подойдёт. Для общего же случая (когда сайт произвольный) надо узнавать кодировку, прежде чем раскодировать, и для этого нужно дополнительный код писать.
Отредактировано py.user.next (Апрель 26, 2020 04:22:32)
Онлайн
- Пожаловаться
- Начало
- » Python для новичков
- » Множественные замены в текстовом файле
