github · gpt-researcher92bfc0388c5f

Evidence

"""Load saved cookies before visiting the target URL"""

        cookie_file = Path(self.cookie_filename)

        if cookie_file.exists():

            cookies = pickle.load(open(self.cookie_filename, "rb"))

            for cookie in cookies:

                self.driver.add_cookie(cookie)

        else:

Remediation

Replace pickle with json/msgpack or a schema-validated format (protobuf, cap'n proto). Use yaml.safe_load instead of yaml.load. Never deserialize data from an untrusted source with these APIs.

Evidence

# Parse XML content

            try:

                root = ET.fromstring(response.text)

                # Extract title

                title = root.find('.//article-title')

Remediation

Use defusedxml in Python (drop-in replacement for stdlib XML modules). In Java, set XMLConstants.FEATURE_SECURE_PROCESSING=true and disable external-DTD features on the factory before parsing.

Evidence

---

sidebar_position: 2

---

# Advanced Usage

This guide covers advanced usage scenarios and configurations for the GPT Researcher MCP Server.

## Custom Configuration

You can customize the MCP server behavior by modifying various configuration parameters:

### Environment Variables

Create a `.env` file with additional configuration options:

```bash

# Required API keys

OPENAI_API_KEY=your_openai_api_key

TAVILY_API_KEY=your_tavily_api_key

# Optional configurations assuming using OpenAI

STRAT

Remediation

Apply auth middleware at the route or router level: - Express / Fastify / Koa: `passport.authenticate(...)`, `requireAuth`, `verifyToken`, or an equivalent JWT middleware applied via `router.use(authMw)` or as a per-route handler. - FastAPI: `Depends(get_current_user)`, `OAuth2PasswordBearer`, `HTTPBearer`, or `verify_jwt` dependency. - Flask: `@login_required`, `@auth_required`, `@jwt_required`, or call `verify_jwt_in_request()` in the handler. Mounting MCP behind a s

Evidence

try:

            if self.is_url():

                try:

                    response = requests.get(self.link, timeout=(5, 30), stream=True)

                    response.raise_for_status()

                except requests.exceptions.SSLError:

                    import logging

                    logging.getLogger(__name__).warning(

                        f"SSL verification failed for {self.link}, retrying without verification"

                    )

                    response = requests.get(se

Remediation

Drop the verify-disable flag. If the peer presents a private CA: - Python: pass `verify="/path/to/ca.pem"` or trust the system store - Node: `new https.Agent({ ca: fs.readFileSync("ca.pem") })` - Go: load the CA via `x509.NewCertPool().AppendCertsFromPEM(...)` and set `tls.Config.RootCAs` Self-signed certificates: import the cert into the OS trust chain rather than disabling verification per-call.

Evidence

TOGETHER_API_KEY=

BING_API_KEY=

HELICONE_API_KEY=

Remediation

Remove the credential from the `.env` file and replace with a template placeholder: OPENAI_API_KEY=<your-openai-key> OPENAI_API_KEY=${OPENAI_API_KEY} Rename the file to `.env.example` so humans know it is a template. Store the real value in a secret manager and inject it at runtime. If the credential has already been committed, revoke it immediately (git history still contains it).

Remediation

Upgrade the pinned dependency to a patched version. Check the CVE's advisory URL for the recommended safe release, or use `npm audit fix` / `pip-audit --fix`. If no patched release is available yet, pin to a known-good prior version, vendor the fix, or remove the dependency.

Remediation

Typosquat: verify you meant the popular package. If so, correct the spelling; if you truly intended the less-common name, suppress with an inline waiver. Stale release: check whether the package has a maintained fork or successor. If no patched release exists, vendor the code or migrate to an active alternative before the unmaintained code accrues unfixed CVEs.

Evidence

###############################################

# 1) Dependencies layer

###############################################

FROM node:18.17.0-alpine AS deps

WORKDIR /app

# Copy only package manifest first for better layer caching

COPY package.json ./

# Install dependencies (no lock file present – recommend adding one for reproducibility)

RUN npm install --legacy-peer-deps

###############################################

# 2) Builder layer – builds Next.js (.next)

##################################

Remediation

Create and switch to a non-root user before the CMD / ENTRYPOINT: RUN adduser --system --uid 1000 app USER 1000 Or reuse the base image's shipped non-root account (e.g. `USER nobody`, `USER nonroot` on distroless). Multi-stage builds only need the USER directive in the final stage.

Evidence

FROM python:3.11-slim

WORKDIR /app

# Copy requirements first to leverage Docker cache

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

# Copy the rest of the application

COPY . .

# Expose the port the app will run on

EXPOSE 8000

# Start the application

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Remediation

Create and switch to a non-root user before the CMD / ENTRYPOINT: RUN adduser --system --uid 1000 app USER 1000 Or reuse the base image's shipped non-root account (e.g. `USER nobody`, `USER nonroot` on distroless). Multi-stage builds only need the USER directive in the final stage.

Evidence

FROM node:18.17.0-alpine

WORKDIR /app

COPY ./package.json ./

RUN npm install --legacy-peer-deps

COPY . .

CMD ["node", "index.js"]

Remediation

Create and switch to a non-root user before the CMD / ENTRYPOINT: RUN adduser --system --uid 1000 app USER 1000 Or reuse the base image's shipped non-root account (e.g. `USER nobody`, `USER nonroot` on distroless). Multi-stage builds only need the USER directive in the final stage.

Evidence

FROM node:18.17.0-alpine

WORKDIR /app

COPY ./package.json ./

RUN npm install --legacy-peer-deps

RUN npm install -g nodemon

COPY . .

CMD ["nodemon", "index.js"]

Remediation

Create and switch to a non-root user before the CMD / ENTRYPOINT: RUN adduser --system --uid 1000 app USER 1000 Or reuse the base image's shipped non-root account (e.g. `USER nobody`, `USER nonroot` on distroless). Multi-stage builds only need the USER directive in the final stage.

Evidence

########################################################################

# Stage 1: Frontend build

########################################################################

FROM node:slim AS frontend-builder

WORKDIR /app/frontend/nextjs

# Copy package files and install dependencies

COPY frontend/nextjs/package.json frontend/nextjs/package-lock.json* ./

RUN npm install --legacy-peer-deps

# Copy the rest of the frontend application and build it

COPY frontend/nextjs/ ./

RUN npm run build

########

Remediation

Create and switch to a non-root user before the CMD / ENTRYPOINT: RUN adduser --system --uid 1000 app USER 1000 Or reuse the base image's shipped non-root account (e.g. `USER nobody`, `USER nonroot` on distroless). Multi-stage builds only need the USER directive in the final stage.

Evidence

FROM node:18.17.0-alpine

WORKDIR /app

COPY ./package.json ./

RUN npm install --legacy-peer-deps

COPY . .

CMD ["npm", "run", "dev"]

Remediation

Create and switch to a non-root user before the CMD / ENTRYPOINT: RUN adduser --system --uid 1000 app USER 1000 Or reuse the base image's shipped non-root account (e.g. `USER nobody`, `USER nonroot` on distroless). Multi-stage builds only need the USER directive in the final stage.

Evidence

continue

            query = example['problem']

            print(f"\nEvaluating query: {query}")

            try:

                result = await evaluate_single_query(query, evaluator)

                results.append(result)

Remediation

Strip C0/C1 control codes before printing user-controlled values. Python: re.sub(r"[\x00-\x08\x0b-\x1f\x7f]", "", s). Prefer a structured logger (json/logfmt) over raw print to stdout.

Evidence

}

          }

        } catch (error) {

          console.error('Error parsing WebSocket message:', error, event.data);

        }

      };

Remediation

Strip C0/C1 control codes before printing user-controlled values. Python: re.sub(r"[\x00-\x08\x0b-\x1f\x7f]", "", s). Prefer a structured logger (json/logfmt) over raw print to stdout.

Evidence

Returns:

        """

        print("Searching with query {0}...".format(self.query))

        """Useful for general internet search queries using the Bing API."""

        # Search the query

Remediation

Strip C0/C1 control codes before printing user-controlled values. Python: re.sub(r"[\x00-\x08\x0b-\x1f\x7f]", "", s). Prefer a structured logger (json/logfmt) over raw print to stdout.

Evidence

print(f"Breadth: {progress.current_breadth}/{progress.total_breadth}")

        print(f"Queries: {progress.completed_queries}/{progress.total_queries}")

        if progress.current_query:

            print(f"Current query: {progress.current_query}")

    # Initialize researcher with deep research type

    researcher = GPTResearcher(

Remediation

Strip C0/C1 control codes before printing user-controlled values. Python: re.sub(r"[\x00-\x08\x0b-\x1f\x7f]", "", s). Prefer a structured logger (json/logfmt) over raw print to stdout.

Evidence

if (this.socket.readyState === WebSocket.OPEN) {

        this.socket.send(payload);

        console.log('Message sent:', payload);

      } else {

        this.socket.onopen = () => {

          this.socket.send(payload);

Remediation

Strip C0/C1 control codes before printing user-controlled values. Python: re.sub(r"[\x00-\x08\x0b-\x1f\x7f]", "", s). Prefer a structured logger (json/logfmt) over raw print to stdout.

Evidence

try {

      body = await request.json();

    } catch (parseError) {

      console.error('Error parsing request body:', parseError);

      return NextResponse.json(

        { error: 'Invalid JSON in request body' },

        { status: 400 }

Remediation

Strip C0/C1 control codes before printing user-controlled values. Python: re.sub(r"[\x00-\x08\x0b-\x1f\x7f]", "", s). Prefer a structured logger (json/logfmt) over raw print to stdout.

Evidence

# Close the loop

                                if not new_loop.is_closed():

                                    new_loop.close()

                            except Exception:

                                pass  # Ignore close errors

                # Run in a thread pool to avoid blocking the main event loop

                with concurrent.futures.ThreadPoolExecutor() as executor:

Remediation

Log the exception at minimum (`logger.exception(e)`), emit a metric, or re-raise if the error is not recoverable. If you genuinely want to ignore an exception, say so with a comment.

Evidence

logger.error(f"Error during WebSocket disconnection: {e}")

            # Still try to close the connection if possible

            try:

                await websocket.close()

            except Exception:

                pass  # If this fails too, there's nothing more we can do

    async def start_streaming(self, task, report_type, report_source, source_urls, document_urls, tone, websocket, headers=None, query_domains=[], mcp_enabled=False, mcp_strategy="fast", mcp_configs=[], max_search_resul

Remediation

Log the exception at minimum (`logger.exception(e)`), emit a metric, or re-raise if the error is not recoverable. If you genuinely want to ignore an exception, say so with a comment.

Evidence

except asyncio.TimeoutError:

                                    logger.debug("Timeout during task cleanup, continuing...")

                                except Exception:

                                    pass  # Ignore other cleanup errors

                        except Exception:

                            pass  # Ignore cleanup errors

                        finally:

                            try:

                                # Give the loop a moment to finish any final cleanup

Remediation

Log the exception at minimum (`logger.exception(e)`), emit a metric, or re-raise if the error is not recoverable. If you genuinely want to ignore an exception, say so with a comment.

Evidence

define(["exports"],function(t){"use strict";try{self["workbox:core:7.0.0"]&&_()}catch(t){}const e=(t,...e)=>{let s=t;return e.length>0&&(s+=` :: ${JSON.stringify(e)}`),s};class s extends Error{constructor(t,s){super(e(t,s)),this.name=t,this.details=s}}try{self["workbox:routing:7.0.0"]&&_()}catch(t){}const n=t=>t&&"object"==typeof t?t:{handle:t};class r{constructor(t,e,s="GET"){this.handler=n(e),this.match=t,this.method=s}setCatchHandler(t){this.catchHandler=n(t)}}class i extends r{constructor(t,

Remediation

Log the exception at minimum (`logger.exception(e)`), emit a metric, or re-raise if the error is not recoverable. If you genuinely want to ignore an exception, say so with a comment.

Evidence

)

                                    )

                                except asyncio.TimeoutError:

                                    logger.debug("Timeout during task cleanup, continuing...")

                                except Exception:

                                    pass  # Ignore other cleanup errors

                        except Exception:

                            pass  # Ignore cleanup errors

                        finally:

Remediation

Log the exception at minimum (`logger.exception(e)`), emit a metric, or re-raise if the error is not recoverable. If you genuinely want to ignore an exception, say so with a comment.