- 添加AK和SK的检测

- 添加检测规则提交入口
- 添加.gitignore文件
- 添加gitee高速下载通道
- 优化txt结果集输出方式
- 去除一些不必要的输出
- 更新apktool至最新版本
- 更新baksmali至最新版本
- 修复单dex文件不解析的问题
- 修复Windows下长文件不解析的问题
- 修复目录中包含空格无法解析的问题
- 修复WEB页面或者目录扫描的问题
- 修复使用默认python执行命令跳出应用商店问题
- 修复iOS应用使用IPA无结果的问题
- 修复因JS文件导致在进行AK搜索时出现的雪崩问题

.gitignore

@@ -0,0 +1,116 @@
+result_*.txt
+result_*.xls
+download/
+history/
+out/
+
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+*.pyc
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+.DS_Store
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+
+# add
+.idea/

CODE_OF_CONDUCT.md

@@ -0,0 +1,128 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity
+and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+blsm@vip.qq.com.
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or
+permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior,  harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
+
+Community Impact Guidelines were inspired by [Mozilla's code of conduct
+enforcement ladder](https://github.com/mozilla/diversity).
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see the FAQ at
+https://www.contributor-covenant.org/faq. Translations are available at
+https://www.contributor-covenant.org/translations.

README.md

@@ -1,4 +1,4 @@
-![Language](https://img.shields.io/badge/Language-Python3-blue) ![License](https://img.shields.io/badge/License-GPL3.0-orange)
+ ![License](https://img.shields.io/badge/Version-V1.0.8-red) ![Language](https://img.shields.io/badge/Language-Python3-blue) ![License](https://img.shields.io/badge/License-GPL3.0-orange)
 
 
 该项目目前仅仅是规划项目中的冰山一角，如果您对此项目感兴趣或者想参与后继项目的开发工作或者翻译工作中，请发送邮件至blsm@vip.qq.com说明你的能力和诉求。
@@ -90,6 +90,11 @@ AppInfoScanner
     或者复制以下链接到浏览器下载最新正式版本
     
     https://github.com/kelvinBen/AppInfoScanner/releases/latest
+
+    国内快速下载通道:
+
+    git clone https://gitee.com/kelvin_ben/AppInfoScanner.git
+
 ```
 
 2. 安装依赖库
@@ -373,6 +378,35 @@ method: 用于配置自动下载过程中需要的请求方法
 
 ```
 请将错误截图以及对应的APK文件提交至 https://github.com/kelvinBen/AppInfoScanner/issues，作者看到后会及时进行处理。
+```
+## 自定义规则添加
+
+自定义规则提交路径：
+
+[点击添加自定义规则](https://github.com/kelvinBen/AppInfoScanner/issues/7）
+
+提交格式：
+```
+1. APP自定义组件添加
+
+如： fastjson的规则如下：
+APP组件: fastjson com.alibaba.fastjson
+
+2. 需要进行搜索的字符串
+
+如：查询阿里的AK规则如下:
+字符串: 
+阿里云AK .*accessKeyId.*".*"
+
+3. 需要搜素的web文件后缀名
+
+如：jsp文件的规则如下：
+网站： java语言 jsp
+
+4. Android壳规则
+如： 某数字公司的壳规则如下：
+壳：某数字公司 com.stub.StubApp
+
 ```
 
 ## 联系作者
@@ -383,7 +417,8 @@ method: 用于配置自动下载过程中需要的请求方法
 
 提交需求、提交BUG修复、技术交流、商务合作均可添加作者好友。
 
-
+## Stargazers
+[![Stargazers over time](https://starchart.cc/kelvinBen/AppInfoScanner.svg)](https://starchart.cc/kelvinBen/AppInfoScanner)
 
 # 404StarLink 2.0 - Galaxy
 ![](https://github.com/knownsec/404StarLink-Project/raw/master/logo.png)

__init__.py

@@ -1,3 +1,4 @@
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Author: kelvinBen
 # Github: https://github.com/kelvinBen/AppInfoScanner

app.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Author: kelvinBen
 # Github: https://github.com/kelvinBen/AppInfoScanner

config.py

@@ -1,3 +1,4 @@
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Author: kelvinBen
 # Github: https://github.com/kelvinBen/AppInfoScanner
@@ -30,7 +31,7 @@ filter_components = [
 filter_strs =[
     r'https://.*|http://.*',
     # r'.*://([[0-9]{1,3}\.]{3}[0-9]{1,3}).*',
-    r'.*://([\d{1,3}\.]{3}\d{1,3}).*'
+    r'.*://([\d{1,3}\.]{3}\d{1,3}).*',
     # r'/[a-z0-9A-Z]+/.*'
 ]
 
@@ -73,7 +74,6 @@ shell_list =[
 web_file_suffix =[
     "html",
     "js",
-    "html",
     "xml",
     "php",
     "jsp",
@@ -85,10 +85,10 @@ web_file_suffix =[
 
 # 配置需要忽略网络嗅探的文件后缀名,此处根据具体需求进行配置，默认为不过滤
 sniffer_filter=[
-    # "jpg",
-    # "png",
-    # "jpeg",
-    # "gif",
+    "jpg",
+    "png",
+    "jpeg",
+    "gif",
 ]
 
 # 配置自动下载Apk文件或者缓存HTML的请求头信息

libs/__init__.py

@@ -1,3 +1,4 @@
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Author: kelvinBen
 # Github: https://github.com/kelvinBen/AppInfoScanner

libs/core/__init__.py

@@ -1,3 +1,4 @@
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Author: kelvinBen
 # Github: https://github.com/kelvinBen/AppInfoScanner
@@ -87,7 +88,14 @@ class Bootstrapper(object):
             print("[*] Create directory {}".format(out_dir))
 
         if os.path.exists(output_path):
-            shutil.rmtree(output_path)
+            try:
+                shutil.rmtree(output_path)
+            except Exception as e:
+                # 解决windows超长文件名删除问题
+                if not (platform.system() == "Windows"):
+                    raise e
+                self.__removed_dirs_cmd__(output_path)
+                
         os.makedirs(output_path)
         print("[*] Create directory {}".format(output_path))
 
@@ -105,9 +113,16 @@ class Bootstrapper(object):
 
         if os.path.exists(xls_result_path):
             os.remove(xls_result_path)
-
-        
-            
-
-
-        
+    
+    def __removed_dirs_cmd__(self,output_path):
+        files = os.listdir(output_path)
+        for file in files:
+            new_dir = os.path.join(output_path,"newdir")
+            old_dir = os.path.join(output_path,file)
+            if not os.path.exists(new_dir):
+                os.makedirs(new_dir)
+            os.chdir(output_path)
+            cmd = ("robocopy %s %s /purge") % (new_dir, old_dir)
+            os.system(cmd)
+            os.removedirs(new_dir)
+            os.removedirs(old_dir)

libs/core/download.py

@@ -1,3 +1,4 @@
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Author: kelvinBen
 # Github: https://github.com/kelvinBen/AppInfoScanner
@@ -12,7 +13,6 @@ import libs.core as cores
 from requests.packages import urllib3
 from requests.adapters import HTTPAdapter    
 
-
 class DownloadThreads(threading.Thread):    
 
     def __init__(self,input_path,file_name,cache_path,types):
@@ -55,7 +55,7 @@ class DownloadThreads(threading.Thread):
                                     sys.stdout.flush()
                         f.close()
                 else:
-                    html = resp.html()
+                    html = resp.text
                     with open(self.cache_path,"w",encoding='utf-8',errors='ignore') as f:
                         f.write(html)
                         f.close()

libs/core/net.py

@@ -1,3 +1,4 @@
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Author: kelvinBen
 # Github: https://github.com/kelvinBen/AppInfoScanner

libs/core/parses.py

@@ -1,3 +1,4 @@
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Author: kelvinBen
 # Github: https://github.com/kelvinBen/AppInfoScanner
@@ -26,22 +27,20 @@ class ParsesThreads(threading.Thread):
             
             file_path = self.file_queue.get(timeout = 5)
             scan_str = ("[+] Scan file : %s" % file_path)
-            print(scan_str)
-
             if self.types == "iOS":
                 self.__get_string_by_iOS__(file_path)
             else:
                 self.__get_string_by_file__(file_path)
             
-            result_set =  set(self.result_list)
-            if len(result_set) !=0:
+            result_set = set(self.result_list)
+            if len(result_set) != 0:
                 self.result_dict[file_path] = result_set
 
     def __get_string_by_iOS__(self,file_path):
         output_path = cores.output_path
         strings_path = cores.strings_path
         temp =  os.path.join(output_path,"temp.txt")
-        cmd_str = ("%s %s > %s") % (strings_path,file_path,temp)
+        cmd_str = ('"%s" "%s" > "%s"') % (str(strings_path),str(file_path),str(temp))
         if os.system(cmd_str) == 0:
             with open(temp,"r",encoding='utf-8',errors='ignore') as f:
                 lines = f.readlines()
@@ -55,6 +54,13 @@ class ParsesThreads(threading.Thread):
             pattern = re.compile(r'\"(.*?)\"') 
             results = pattern.findall(file_content)
 
+            # 搜素AK和SK信息
+            if not ".js" == file_path[-3:]:
+                akAndSkList = re.compile(r'.*accessKeyId.*".*"|.*accessKeySecret.*".*"|.*secret.*".*"').findall(file_content)
+                for akAndSk in akAndSkList:
+                    self.result_list.append(akAndSk.strip())
+                    print("[+] AK or SK in:",akAndSk.strip())
+
             # 遍历所有的字符串
             for result in set(results): 
                 self.__parse_string__(result)    
@@ -64,7 +70,6 @@ class ParsesThreads(threading.Thread):
         for filter_str in config.filter_strs:
             filter_str_pat = re.compile(filter_str) 
             filter_resl = filter_str_pat.findall(result)
-
             # 过滤掉未搜索到的内容
             if len(filter_resl)!=0:
                 # 提取第一个结果
@@ -100,4 +105,4 @@ class ParsesThreads(threading.Thread):
 
     def run(self):
         self.threadLock = threading.Lock()
-        self.__regular_parse__()
+        self.__regular_parse__()

libs/task/__init__.py

@@ -1,3 +1,4 @@
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Author: kelvinBen
 # Github: https://github.com/kelvinBen/AppInfoScanner

libs/task/android_task.py

@@ -1,3 +1,4 @@
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Author: kelvinBen
 # Github: https://github.com/kelvinBen/AppInfoScanner
@@ -57,6 +58,8 @@ class AndroidTask(object):
             dex_md5 = md5_obj.hexdigest().lower()
             self.file_identifier.append(dex_md5)
             output_path = os.path.join(base_out_path,dex_md5)
+            if not os.path.exists(output_path):
+                os.makedirs(output_path)
             self.__decode_dex__(file_path,backsmali_path,output_path)
         else:
             return "error"
@@ -73,7 +76,7 @@ class AndroidTask(object):
 
     # 分解apk
     def __decode_apk__(self,file_path,apktool_path,output_path):
-        cmd_str = ("java -jar %s d -f %s -o %s --only-main-classe") % (apktool_path,str(file_path),output_path)
+        cmd_str = ('java -jar "%s" d -f "%s" -o "%s" --only-main-classe') % (str(apktool_path),str(file_path),str(output_path))
         if os.system(cmd_str) == 0:
             self.__shell_test__(output_path)
             self.__scanner_file_by_apktool__(output_path)
@@ -84,7 +87,7 @@ class AndroidTask(object):
 
     # 分解dex
     def __decode_dex__(self,file_path,backsmali_path,output_path):
-        cmd_str = ("java -jar %s d %s") % (backsmali_path,str(file_path))
+        cmd_str = ('java -jar "%s" d "%s"') % (str(backsmali_path),str(file_path))
         if os.system(cmd_str) == 0:
             self.__get_scanner_file__(output_path)
         else:

libs/task/base_task.py

@@ -1,3 +1,4 @@
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Author: kelvinBen
 # Github: https://github.com/kelvinBen/AppInfoScanner
@@ -100,6 +101,7 @@ class BaseTask(object):
     def __print_control__(self,packagename,comp_list,file_identifier):
         txt_result_path = cores.txt_result_path
         xls_result_path = cores.xls_result_path
+        all_flag = cores.all_flag
                 
         if self.sniffer:
             print("[*] ========= Sniffing the URL address of the search ===============")
@@ -114,11 +116,21 @@ class BaseTask(object):
             for json in comp_list:
                 print(json)
         
-        if cores.all_flag:
-            print("[*] For more information about the search, see TXT file result: %s" %(cores.txt_result_path))
+        if all_flag:
+            value_list = []
+            with open(txt_result_path,"a+",encoding='utf-8',errors='ignore') as f:
+                for key,value in self.result_dict.items():
+                    f.write(key+"\r")
+                    for result in value:
+                        if result in value_list:
+                            continue
+                        value_list.append(result)
+                        f.write("\t"+result+"\r")
+                f.close()
+            print("[*] For more information about the search, see TXT file result: %s" %(txt_result_path))
 
         if self.sniffer:
-            print("[*] For more information about the search, see XLS file result: %s" %(cores.xls_result_path))
+            print("[*] For more information about the search, see XLS file result: %s" %(xls_result_path))
 
     def __history_handle__(self):
         domain_history_path =  cores.domain_history_path

libs/task/download_task.py

@@ -1,3 +1,4 @@
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Author: kelvinBen
 # Github: https://github.com/kelvinBen/AppInfoScanner

libs/task/ios_task.py

@@ -1,3 +1,4 @@
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Author: kelvinBen
 # Github: https://github.com/kelvinBen/AppInfoScanner
@@ -10,7 +11,6 @@ import platform
 import libs.core as cores
 from queue import Queue
 
-
 class iOSTask(object):
     elf_file_name = ""
     def __init__(self,path):
@@ -87,7 +87,6 @@ class iOSTask(object):
         with zipfile.ZipFile(self.path,"r") as zip_files:
             zip_file_names = zip_files.namelist()
             zip_files.extract(zip_file_names[0],output_path)
-            
             try:
                 new_zip_file =  zip_file_names[0].encode('cp437').decode('utf-8')
             except UnicodeEncodeError:
@@ -96,8 +95,8 @@ class iOSTask(object):
                 old_zip_dir = self.__get_parse_dir__(output_path,zip_file_names[0])
                 new_zip_dir = self.__get_parse_dir__(output_path,new_zip_file)
                 os.rename(old_zip_dir,new_zip_dir)
+            
             for zip_file in zip_file_names:
-               
                 old_ext_path = zip_files.extract(zip_file,output_path)
                 start = str(old_ext_path).index("Payload")
                 dir_path = old_ext_path[start:len(old_ext_path)]
@@ -105,7 +104,7 @@ class iOSTask(object):
                 try:
                     new_zip_file = zip_file.encode('cp437').decode('utf-8')
                 except UnicodeEncodeError:
-                   new_zip_file = zip_file.encode('utf-8').decode('utf-8')
+                    new_zip_file = zip_file.encode('utf-8').decode('utf-8')
                 
                 new_ext_path = os.path.join(output_path,new_zip_file)
 
@@ -117,6 +116,8 @@ class iOSTask(object):
                     if not os.path.exists(dir_path):
                         os.makedirs(dir_path)
                 shutil.move(old_ext_path, new_ext_path)
+                if os.path.exists(old_ext_path):
+                    os.remove(old_ext_path)
 
 
     def __get_parse_dir__(self,output_path,file_path):

libs/task/net_task.py

@@ -1,3 +1,4 @@
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Author: kelvinBen
 # Github: https://github.com/kelvinBen/AppInfoScanner
@@ -55,53 +56,45 @@ class NetTask(object):
         txt_result_path = cores.txt_result_path
         append_file_flag = True
         
-        with open(txt_result_path,"a+",encoding='utf-8',errors='ignore') as f:
-            for key,value in self.result_dict.items():
-                if cores.all_flag:
-                    f.write(key+"\r")
-                    
-                for result in value:
-                    if result in self.value_list:
-                        continue
-                    self.value_list.append(result)
-                    if cores.all_flag:
-                        f.write("\t"+result+"\r")
+        for key,value in self.result_dict.items():
+            for result in value:
+                if result in self.value_list:
+                    continue
+                self.value_list.append(result)
 
-                    if (("http://" in result) or ("https://" in result)) and ("." in result):
-                        domain = result.replace("https://","").replace("http://","")
+                if (("http://" in result) or ("https://" in result)) and ("." in result):
+                    domain = result.replace("https://","").replace("http://","")
                         
-                        if "{" in result or "}" in result or "[" in result or "]" in result or "\\" in result or "!" in result or "," in result:
-                            continue
+                    if "{" in result or "}" in result or "[" in result or "]" in result or "\\" in result or "!" in result or "," in result:
+                        continue
 
-                        if "/" in domain:
-                            domain = domain[:domain.index("/")]
+                    if "/" in domain:
+                        domain = domain[:domain.index("/")]
                         
-                        if "|" in result:
-                            result = result[:result.index("|")]
-                        # 目前流通的域名中加上协议头最短长度为11位
-                        if len(result) <= 10:
-                            continue
+                    if "|" in result:
+                        result = result[:result.index("|")]
+                    # 目前流通的域名中加上协议头最短长度为11位
+                    if len(result) <= 10:
+                        continue
                         
-                        url_suffix = result[result.rindex(".")+1:].lower()
-                        if not(cores.resource_flag and url_suffix in config.sniffer_filter):
-                            self.domain_queue.put({"domain":domain,"url_ip":result})
+                    url_suffix = result[result.rindex(".")+1:].lower()
+                    if not(cores.resource_flag and url_suffix in config.sniffer_filter):
+                        self.domain_queue.put({"domain":domain,"url_ip":result})
                         
-                        for identifier in self.file_identifier:
-                            if identifier in self.app_history_list:
-                                if not(domain in self.domain_history_list): 
-                                    self.domain_list.append(domain)
-                                    self.__write_content_in_file__(cores.domain_history_path,domain)
-                                continue
-
-                            if not(domain in self.domain_list):
+                    for identifier in self.file_identifier:
+                        if identifier in self.app_history_list:
+                            if not(domain in self.domain_history_list): 
                                 self.domain_list.append(domain)
                                 self.__write_content_in_file__(cores.domain_history_path,domain)
+                            continue
 
-                            if append_file_flag:
-                                self.__write_content_in_file__(cores.app_history_path,identifier)
-                                append_file_flag = False
-                    
-            f.close()
+                        if not(domain in self.domain_list):
+                            self.domain_list.append(domain)
+                            self.__write_content_in_file__(cores.domain_history_path,domain)
+
+                        if append_file_flag:
+                            self.__write_content_in_file__(cores.app_history_path,identifier)
+                            append_file_flag = False
 
     def __start_threads__(self,worksheet):
         for threadID in range(0,self.threads) : 

libs/task/web_task.py

@@ -1,3 +1,4 @@
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Author: kelvinBen
 # Github: https://github.com/kelvinBen/AppInfoScanner
@@ -13,6 +14,7 @@ class WebTask(object):
     def __init__(self, path):
         self.path = path
         self.file_queue = Queue()
+        self.file_identifier = []
 
     def start(self):
         if len(config.web_file_suffix) <=0:
@@ -26,7 +28,7 @@ class WebTask(object):
                 err_info = ("Retrieval of this file type is not supported. Select a file or directory with a suffix of %s" % ",".join(scanner_file_suffix))
                 raise Exception(err_info)
             self.file_queue.put(self.path)
-        return {"comp_list":[],"shell_flag":False,"file_queue":self.file_queue,"packagename":None}
+        return {"comp_list":[],"shell_flag":False,"file_queue":self.file_queue,"packagename":None,"file_identifier":self.file_identifier}
 
     def __get_scanner_file__(self,scanner_dir,file_suffix):
         dir_or_files = os.listdir(scanner_dir)

update.md

@@ -1,3 +1,13 @@
+### V1.0.8
+- 添加AK和SK的检测
+- 添加检测规则提交入口
+- 添加.gitignore文件
+- 优化txt结果集输出方式
+- 修复目录中包含空格无法解析的问题
+- 修复WEB页面或者目录扫描的问题
+- 修复使用默认python执行命令跳出应用商店问题
+- 修复iOS应用使用IPA无结果的问题
+
 ### V1.0.7
 - 新增文件自动下载功能，支持APK文件,非AppStore的IPA文件的下载以及对H5或者html页面进行缓存
 - 新增后缀信息修正功能，误填任务类型可以根据后缀名进行自我修正
@@ -11,7 +21,6 @@
 - 修复以目录的方式对Android反编译原目录被覆盖的问题
 - 修复AI分析完毕后不能很好过滤的问题
 
-
 ### V1.0.6
 - 新增AI智能分析快速过滤第三方URL地址
 - 新增Domain\Title\CDN识别功能
@@ -54,7 +63,6 @@
 - 新增IPA相关静态扫描规则支持
 - 新增对Android加固信息特征识别
 
-
 ### V1.0.0
 - 支持目录批量扫描
 - 支持DEX、APK、Smali、HTML、JS、等文件的静态资源采集
@@ -63,4 +71,4 @@
 - 支持URL地址信息采集
 - 支持中间件信息采集
 - 支持多线程
-- 支持忽略资源文件采集
+- 支持忽略资源文件采集