Deliver web by SMS

Deliver web by SMS [∞]

This is a creative solution to provide web content in regions with limited internet connectivity. Below is a high-level design of your system, considering simplicity, scalability, and functionality.

---

System Design Overview

1. User Interaction

• Input: Users send an SMS containing a request (e.g., a URL or query) to a specific phone number.
• Output: The user receives an SMS with the requested web content in a reduced, SMS-friendly format.

2. Backend Workflow

• SMS Gateway:
  • Receives the SMS request from the user.
  • Forwards the message to your web server via a secure API or webhook.
• Web Server:
  • Processes the request to fetch the desired web page.
  • Strips the HTML overhead, leaving only essential content (e.g., text, simplified formatting, and URLs as plain text).
  • Returns the simplified content to the SMS gateway.
• SMS Gateway:
  • Sends the simplified content back to the user's phone.

---

Key Components

1. SMS Gateway

• Description: Acts as a bridge between users and your web server.
• Options: Use services like Twilio, Vonage, or local SMS gateway providers.
• Features:
  • Handle incoming SMS requests.
  • Relay the request to your backend web server.
  • Send responses from the web server back to the user.

2. Web Server

• Tech Stack:
  • Use a lightweight framework (e.g., Flask, FastAPI, or Node.js) for efficient processing.
• Tasks:
  1. Request Processing:
     • Parse the SMS to extract the URL or query.
  2. Content Retrieval:
     • Fetch the webpage using an HTTP client (e.g., Python’s requests or Node.js’s axios).
  3. HTML Reduction:
     • Use an HTML parser (e.g., BeautifulSoup in Python) to strip unnecessary tags (e.g., <script>, <style>).
     • Extract essential elements such as:
       • Plain text content (<p>, <h1>, <h2>, etc.)
       • Plain links (<a> as text: [URL]).
  4. SMS Formatting:
     • Truncate content to fit SMS length limits (160 characters per SMS).
     • For long content, split into multiple SMS messages.
• Output: Relay the simplified content back to the SMS gateway.

3. Database (Optional)

• Purpose: Cache frequently requested pages to reduce load and latency.
• Implementation: Use lightweight storage (e.g., SQLite, Redis).

4. SMS Content Optimization

• Use plain text.
• Strip redundant information (e.g., images, styles, and scripts).
• Provide essential navigation or summary details (e.g., “Reply with 1 for more details about [Topic]”).

5. Security

• Sanitize URLs to prevent malicious input.
• Implement rate-limiting to prevent abuse.
• Encrypt data between the SMS gateway and your web server.

---

Example Workflow

1. User Request: Sends URL: example.com via SMS.
2. SMS Gateway:
   • Receives the SMS.
   • Forwards example.com to the web server.
3. Web Server:
   • Fetches example.com.
   • Extracts relevant text and links.
   • Formats it into SMS-friendly chunks:
     • SMS 1: Example.com - Title: Learn Coding. Intro: Free courses. More: [URL].
     • SMS 2: Details: Visit: example.com/details or reply 1 for summary.
   • Returns content to the SMS gateway.
4. SMS Gateway: Sends SMS back to the user.

---

Implementation Challenges

1. SMS Length Limitation:
   • SMS length is limited to 160 characters. Use concatenated SMS for longer messages.
2. Content Complexity:
   • Not all web pages are easily simplified. Prioritize extracting key information (text, links).
3. Cost:
   • Outbound SMS costs can escalate. Consider funding models or sponsorships.

---

from flask import Flask, request, jsonify

import requests

from bs4 import BeautifulSoup

app = Flask(__name__)

# Facebook API credentials

PAGE_ACCESS_TOKEN = 'your_page_access_token'

# Function to send messages via Messenger

def send_message(recipient_id, message_text):

   url = f"https://graph.facebook.com/v11.0/me/messages?access_token={PAGE_ACCESS_TOKEN}"

   headers = {'Content-Type': 'application/json'}

   payload = {

       'recipient': {'id': recipient_id},

       'message': {'text': message_text}

   }

   response = requests.post(url, json=payload, headers=headers)

   return response.json()

# Function to fetch and simplify web content

def fetch_web_content(url):

   try:

       response = requests.get(url, timeout=5)

       response.raise_for_status()

       soup = BeautifulSoup(response.text, 'html.parser')

       # Extract title and paragraph text

       title = soup.title.string if soup.title else "No Title"

       paragraphs = soup.find_all('p')

       text_content = " ".join(p.get_text(strip=True) for p in paragraphs[:3])  # Limit to 3 paragraphs

       # Simplify links

       links = soup.find_all('a', href=True)

       links_text = [f"{a.get_text(strip=True)}: {a['href']}" for a in links[:3]]  # Limit to 3 links

       # Prepare summary

       summary = f"Title: {title}\nContent: {text_content}\nLinks:\n" + "\n".join(links_text)

       return summary[:2000]  # Limit to 2000 characters for Messenger

   except Exception as e:

       return f"Error fetching content: {e}"

# Webhook to handle incoming messages

@app.route('/webhook', methods=['POST'])

def webhook():

   data = request.json

   if data['object'] == 'page':

       for entry in data['entry']:

           for messaging_event in entry['messaging']:

               sender_id = messaging_event['sender']['id']

               if 'message' in messaging_event and 'text' in messaging_event['message']:

                   user_message = messaging_event['message']['text']

                   

                   # Fetch and simplify web content

                   if user_message.startswith("URL:"):

                       url = user_message[4:].strip()

                       content = fetch_web_content(url)

                   else:

                       content = "Send a message in the format 'URL:<website>' to fetch content."

                   # Respond to user

                   send_message(sender_id, content)

   return "ok", 200

# Facebook verification endpoint

@app.route('/webhook', methods=['GET'])

def verify_webhook():

   VERIFY_TOKEN = 'your_verify_token'

   mode = request.args.get('hub.mode')

   token = request.args.get('hub.verify_token')

   challenge = request.args.get('hub.challenge')

   if mode == 'subscribe' and token == VERIFY_TOKEN:

       return challenge, 200

   else:

       return 'Forbidden', 403

if __name__ == '__main__':

   app.run(port=5000)