1.Failed operation _store_result. Retrying 2 more times.
我们线上用的celeryexecutor,当我的executor接到队列任务执行任务时,经常会出现上面截图的错误。
经过网上查询发现,问题原因应该是celery无法共用airflow的mysql连接配置导致的,所以需要修改celery的数据库配置,比如增加连接池等等
查到的网上其他的相同问题的链接
https://blog.csdn.net/chanyue123/article/details/120544718
https://jiuaidu.com/jianzhan/616149/
解决方案
1.将本地airflow下的default_celery.py拷贝到airflow的dags文件夹下。
2.增加default_celery.py中的连接池配置
2.修改airflow.cfg
celery_config_options = default_celery.DEFAULT_CELERY_CONFIG
2.airflow db init时报错
ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2k-fips 26 Jan 2017. See: https://github.com/urllib3/urllib3/issues/2168
解决方案
pip3 install urllib3==1.26.15
pip3 list
3.SQLite C Library版本较低
airflow Traceback (most recent call last): File "/home/hadoop/.local/bin/airflow", line 5, in from airflow.main import main File "/home/hadoop/.local/lib/python3.8/site-packages/airflow/init.py", line 34, in conf.validate() File "/home/hadoop/.local/lib/python3.8/site-packages/airflow/configuration.py", line 201, in validate self._validate_config_dependencies() File "/home/hadoop/.local/lib/python3.8/site-packages/airflow/configuration.py", line 241, in _validate_config_dependencies raise AirflowConfigException( airflow.exceptions.AirflowConfigException: error: sqlite C library version too old (< 3.15.0). See
解决方案
wget https://www.sqlite.org/2019/sqlite-autoconf-3290000.tar.gz
tar zxvf sqlite-autoconf-3290000.tar.gz
cd sqlite-autoconf-3290000/
./configure --prefix=/usr/local
make && make install
mv /usr/bin/sqlite3 /usr/bin/sqlite3_old
ln -s /usr/local/bin/sqlite3 /usr/bin/sqlite3
echo "/usr/local/lib" > /etc/ld.so.conf.d/sqlite3.conf
ldconfig
sqlite3 -version
--显示以下,说明安装正确
3.29.0 2019-07-10 17:32:03 fc82b73eaac8b36950e527f12c4b5dc1e147e6f4ad2217ae43ad82882a88bfa6