Hi!
Installed scrapy and just trying to start the tutorial gives me that:
scrapy startproject tutorial Traceback (most recent call last): File "/home/fred/.local/bin/scrapy", line 5, in <module> from scrapy.cmdline import execute File "/home/fred/.local/lib/python3.12/site-packages/scrapy/__init__.py", line 12, in <module> from scrapy.http import FormRequest, Request File "/home/fred/.local/lib/python3.12/site-packages/scrapy/http/__init__.py", line 10, in <module> from scrapy.http.request.form import FormRequest File "/home/fred/.local/lib/python3.12/site-packages/scrapy/http/request/form.py", line 19, in <module> from parsel.selector import create_root_node File "/home/fred/.local/lib/python3.12/site-packages/parsel/__init__.py", line 16, in <module> from parsel.selector import Selector, SelectorList # NOQA ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fred/.local/lib/python3.12/site-packages/parsel/selector.py", line 31, in <module> from .csstranslator import GenericTranslator, HTMLTranslator File "/home/fred/.local/lib/python3.12/site-packages/parsel/csstranslator.py", line 8, in <module> from cssselect.parser import Element, FunctionalPseudoElement, PseudoElement ImportError: cannot import name 'PseudoElement' from 'cssselect.parser' (/usr/lib/python3.12/site-packages/cssselect/parser.py) fred@romeo:~/Downloads/scrapy$ scrapy startproject test Traceback (most recent call last): File "/home/fred/.local/bin/scrapy", line 5, in <module> from scrapy.cmdline import execute File "/home/fred/.local/lib/python3.12/site-packages/scrapy/__init__.py", line 12, in <module> from scrapy.http import FormRequest, Request File "/home/fred/.local/lib/python3.12/site-packages/scrapy/http/__init__.py", line 10, in <module> from scrapy.http.request.form import FormRequest File "/home/fred/.local/lib/python3.12/site-packages/scrapy/http/request/form.py", line 19, in <module> from parsel.selector import create_root_node File "/home/fred/.local/lib/python3.12/site-packages/parsel/__init__.py", line 16, in <module> from parsel.selector import Selector, SelectorList # NOQA ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fred/.local/lib/python3.12/site-packages/parsel/selector.py", line 31, in <module> from .csstranslator import GenericTranslator, HTMLTranslator File "/home/fred/.local/lib/python3.12/site-packages/parsel/csstranslator.py", line 8, in <module> from cssselect.parser import Element, FunctionalPseudoElement, PseudoElement ImportError: cannot import name 'PseudoElement' from 'cssselect.parser' (/usr/lib/python3.12/site-packages/cssselect/parser.py)
Any idea why it's not working?
Thank you.
Fred
On Wed, Feb 7, 2024 at 2:11 AM Frederic Muller fred@cm17.com wrote:
Hi!
Installed scrapy and just trying to start the tutorial gives me that:
scrapy startproject tutorial
[...]
/home/fred/.local/lib/python3.12/site-packages/parsel/csstranslator.py",
line 8, in <module>
from cssselect.parser import Element, FunctionalPseudoElement,PseudoElement ImportError: cannot import name 'PseudoElement' from 'cssselect.parser' (/usr/lib/python3.12/site-packages/cssselect/parser.py) Traceback (most recent call last):
Any idea why it's not working?
Looks like https://stackoverflow.com/questions/77476433/importerror-cannot-import-name-... Fedora 39 has: python-cssselect-1.1.0-6 Have a look at the https://github.com/scrapy/scrapy/issues
You should give enough details so others can reproduce the issue: how did you install scrapy, which version of Fedora, scrapy, and python-cssselect do you have?
There are problems with the Fedora package:
Available Packages Name : python3-scrapy Version : 2.10.1 Release : 1.fc39 Architecture : noarch Size : 659 k Source : python-scrapy-2.10.1-1.fc39.src.rpm Repository : fedora Summary : A high-level Python Screen Scraping framework URL : https://scrapy.org License : BSD Description : Scrapy is a fast high-level screen scraping and web crawling : framework, used to crawl websites and extract structured data : from their pages. It can be used for a wide range of purposes, : from data mining to monitoring and automated testing.
Here, on Fedora 39:
% doas dnf install python3-scrapy doas (gnw3@imacf39) password: Last metadata expiration check: 0:04:29 ago on Wed Feb 7 10:39:54 2024. Error: Problem: conflicting requests - nothing provides (python3.12dist(twisted) < 23.8~~ with python3.12dist(twisted) >= 18.9) needed by python3-scrapy-2.10.1-1.fc39.noarch from fedora (try to add '--skip-broken' to skip uninstallable packages)
twisted has https://nvd.nist.gov/vuln/detail/CVE-2023-46137 so
Many large Python systems have not been ported to python-3.12. I've had success using fedora's python-3.11 in a conda environment. Anaconda python is also a way to run stuff that still needs python 3.10 or 3.11.
On 7 Feb 2024, at 15:10, George N. White III gnwiii@gmail.com wrote:
twisted has https://nvd.nist.gov/vuln/detail/CVE-2023-46137 so
Which is fixed in the version in Fedora as far as I can tell.
Barry
On 7 Feb 2024, at 18:06, Barry barry@barrys-emacs.org wrote:
On 7 Feb 2024, at 15:10, George N. White III gnwiii@gmail.com wrote:
twisted has https://nvd.nist.gov/vuln/detail/CVE-2023-46137 so
Which is fixed in the version in Fedora as far as I can tell
Oh sorry George, I see that scrapy wants the buggy version.
Barry
On 07/02/2024 22:09, George N. White III wrote:
On Wed, Feb 7, 2024 at 2:11 AM Frederic Muller fred@cm17.com wrote:
Hi! Installed scrapy and just trying to start the tutorial gives me that: scrapy startproject tutorial[...]
/home/fred/.local/lib/python3.12/site-packages/parsel/csstranslator.py", line 8, in <module> from cssselect.parser import Element, FunctionalPseudoElement, PseudoElement ImportError: cannot import name 'PseudoElement' from 'cssselect.parser' (/usr/lib/python3.12/site-packages/cssselect/parser.py) Traceback (most recent call last): Any idea why it's not working?Looks like https://stackoverflow.com/questions/77476433/importerror-cannot-import-name-... Fedora 39 has: python-cssselect-1.1.0-6 Have a look at the https://github.com/scrapy/scrapy/issues
You should give enough details so others can reproduce the issue: how did you install scrapy, which version of Fedora, scrapy, and python-cssselect do you have?
There are problems with the Fedora package:
Available Packages Name : python3-scrapy Version : 2.10.1 Release : 1.fc39 Architecture : noarch Size : 659 k Source : python-scrapy-2.10.1-1.fc39.src.rpm Repository : fedora Summary : A high-level Python Screen Scraping framework URL : https://scrapy.org License : BSD Description : Scrapy is a fast high-level screen scraping and web crawling : framework, used to crawl websites and extract structured data : from their pages. It can be used for a wide range of purposes, : from data mining to monitoring and automated testing.
Here, on Fedora 39:
% doas dnf install python3-scrapy doas (gnw3@imacf39) password: Last metadata expiration check: 0:04:29 ago on Wed Feb 7 10:39:54 2024. Error: Problem: conflicting requests - nothing provides (python3.12dist(twisted) < 23.8~~ with python3.12dist(twisted) >= 18.9) needed by python3-scrapy-2.10.1-1.fc39.noarch from fedora (try to add '--skip-broken' to skip uninstallable packages)
twisted has https://nvd.nist.gov/vuln/detail/CVE-2023-46137 so
Many large Python systems have not been ported to python-3.12. I've had success using fedora's python-3.11 in a conda environment. Anaconda python is also a way to run stuff that still needs python 3.10 or 3.11.
So yes... just tried something totally different (opening an IFC file with FreeCAD) and the importer stops saying:
" File "/app/lib/python3.11/site-packages/ifcopenshell/__init__.py", line 62, in <module> raise ImportError("IfcOpenShell not built for '%s'" % python_distribution) <class 'ImportError'>: IfcOpenShell not built for 'linux/64bit/python3.11' "
Back to scrapy I am using F39 updated to the latest everything, I installed using the given instructions on their website which is pip install scrapy scrapy is version 2.11.0 cssselect is version 1.1.0
So should I file a bug somewhere or patiently wait (it's been like for over a month at least)?
Thank you.
Fred
On Thu, Feb 8, 2024 at 12:48 AM Frederic Muller fred@cm17.com wrote:
On 07/02/2024 22:09, George N. White III wrote:
On Wed, Feb 7, 2024 at 2:11 AM Frederic Muller fred@cm17.com wrote:
Hi!
Installed scrapy and just trying to start the tutorial gives me that:
scrapy startproject tutorial
[...]
/home/fred/.local/lib/python3.12/site-packages/parsel/csstranslator.py",
line 8, in <module>
from cssselect.parser import Element, FunctionalPseudoElement,PseudoElement ImportError: cannot import name 'PseudoElement' from 'cssselect.parser' (/usr/lib/python3.12/site-packages/cssselect/parser.py) Traceback (most recent call last):
Any idea why it's not working?
Looks like https://stackoverflow.com/questions/77476433/importerror-cannot-import-name-... Fedora 39 has: python-cssselect-1.1.0-6 Have a look at the https://github.com/scrapy/scrapy/issues
You should give enough details so others can reproduce the issue: how did you install scrapy, which version of Fedora, scrapy, and python-cssselect do you have?
There are problems with the Fedora package:
Available Packages Name : python3-scrapy Version : 2.10.1 Release : 1.fc39 Architecture : noarch Size : 659 k Source : python-scrapy-2.10.1-1.fc39.src.rpm Repository : fedora Summary : A high-level Python Screen Scraping framework URL : https://scrapy.org License : BSD Description : Scrapy is a fast high-level screen scraping and web crawling : framework, used to crawl websites and extract structured data : from their pages. It can be used for a wide range of purposes, : from data mining to monitoring and automated testing.
Here, on Fedora 39:
% doas dnf install python3-scrapy doas (gnw3@imacf39) password: Last metadata expiration check: 0:04:29 ago on Wed Feb 7 10:39:54 2024. Error: Problem: conflicting requests
- nothing provides (python3.12dist(twisted) < 23.8~~ with
python3.12dist(twisted) >= 18.9) needed by python3-scrapy-2.10.1-1.fc39.noarch from fedora (try to add '--skip-broken' to skip uninstallable packages)
twisted has https://nvd.nist.gov/vuln/detail/CVE-2023-46137 so
Many large Python systems have not been ported to python-3.12. I've had success using fedora's python-3.11 in a conda environment. Anaconda python is also a way to run stuff that still needs python 3.10 or 3.11.
So yes... just tried something totally different (opening an IFC file with FreeCAD) and the importer stops saying:
" File "/app/lib/python3.11/site-packages/ifcopenshell/__init__.py", line 62, in <module> raise ImportError("IfcOpenShell not built for '%s'" % python_distribution) <class 'ImportError'>: IfcOpenShell not built for 'linux/64bit/python3.11' "
You don't mention how you installed python-3.11 and FreeCAD. Anaconda provides many python packages that are built in a way that is compatible with their python binaries. Conda manages dependencies. Fedora has /usr/bin/python3.11 from python3.11-3.11.7-2.fc39.x86_64.
Back to scrapy I am using F39 updated to the latest everything, I installed
using the given instructions on their website which is pip install scrapy scrapy is version 2.11.0 cssselect is version 1.1.
So should I file a bug somewhere or patiently wait (it's been like for over
a month at least)?
Python 3.12 requires major changes to some packages, and many developers aren't paid for their contributions, so as long as people are able to continue using affected packages with Python 3.10 or 3.11 there is no urgency to make things work with Python 3.12.
For scrapy there is the added complication of the recent CVE for twisted: do we get patches for older versions of twisted or does scrapy have to be changed to use the current version?
I think the scrapy github Issues already has similar reports. You may want to add your configuration to the existing issue. Post on mail lists rarely get into search results, so github Issues are more likely to help others who have a problem..
On 8 Feb 2024, at 12:38, George N. White III gnwiii@gmail.com wrote:
For scrapy there is the added complication of the recent CVE for twisted: do we get patches for older versions of twisted or does scrapy have to be changed to use the current version?
I don’t think twisted back ports fixes. But it should not be hard, usually, to upgrade to latest twisted. May well just work.
Barry
-
Barry -
Frederic Muller -
George N. White III