fix: 防止重复推送通知 — 添加已发送检查 + GXGP 标记已发送

- Pipeline 新增 _exclude_sent() 方法，通知前查询数据库跳过 is_sent=True 的公告 - GXGP Spider mark_sent 改为 True，配合去重逻辑防止每次爬取重复通知
2026-05-09 16:00:04 +08:00
parent 6fbdf097b3
commit d851dafea9
4 changed files with 48 additions and 20 deletions
@@ -25,7 +25,7 @@ class GXGPSpider(BaseSpider):
            keywords=list(settings.crawler_keywords),
            dedup_enabled=True,
            notify_mode="filtered",
-            mark_sent=False,
+            mark_sent=True,
        )

    async def crawl(self, sources: list[str] | None = None,