Pages:
Author

Topic: Ninjastic.space - BitcoinTalk Post/Address archive + API (Read 16797 times)

legendary
Activity: 2380
Merit: 17063
Fully fledged Merit Cycler - Golden Feather 22-23
 Tongue
I edited the above post, hoping for a review.
The code was already in English, but I translated all the variables' names into English.
I also tried perplexity ai, but it failed miserably as well.
 Looks like i need to learn Pytohn the hard old way.
legendary
Activity: 2758
Merit: 6830
I tried to use chat GPT to use the API and I got a very bad answer:

Quote
Write state-of-the-art, well-commented code  to use ninjastic  API (documentation at https://docs.ninjastic.space )to query the database to get a list of users who wrote on the Italian board (sub-boards included) from midnight of the first day of the previous day to midnight of the first day of the current month. For example on 25 December they should query from midnight on  the first of November to the first of December at midnight

After a few correction and iterations i got this not working code.

Code:

import requests
from datetime import datetime, timedelta

# Costante per il URL base dell'API Ninjastic
BASE_URL = "https://api.ninjastic.space"

# Funzione di supporto per calcolare l'intervallo di date
def calcola_intervallo_date():
    """
    Calcola l'intervallo di date dalla mezzanotte del primo giorno del mese precedente
    fino alla mezzanotte del primo giorno del mese corrente.

    Ritorna:
        start_date (str): La data di inizio in formato ISO 8601.
        end_date (str): La data di fine in formato ISO 8601.
    """
    oggi = datetime.now()
    primo_giorno_mese_corrente = datetime(oggi.year, oggi.month, 1)
    primo_giorno_mese_precedente = (primo_giorno_mese_corrente - timedelta(days=1)).replace(day=1)

    data_inizio = primo_giorno_mese_precedente.isoformat()  # Rimuove 'Z'
    data_fine = primo_giorno_mese_corrente.isoformat()      # Rimuove 'Z'

    return data_inizio, data_fine

# Funzione per ottenere gli utenti e il numero di messaggi dal board italiano
def ottieni_utenti_e_messaggi_board_italiano():
    """
    Esegue una query sull'API Ninjastic per ottenere una lista di utenti che hanno scritto sul board italiano
    (inclusi i sotto-board) e il numero totale di messaggi per ciascuno nell'intervallo di date specificato.

    Ritorna:
        dict: Un dizionario con gli utenti come chiavi e il numero di messaggi come valori.
    """
    data_inizio, data_fine = calcola_intervallo_date()

    # Endpoint e parametri per la richiesta API
    endpoint = f"{BASE_URL}/search"
    params = {
        "after_date": data_inizio,
        "before_date": data_fine,
        "board": 28,  # ID del board italiano
        "child_boards": "true"  # Includi i sotto-board
    }

    try:
        print(f"Invio richiesta a: {endpoint}")
        print(f"Parametri: {params}")

        risposta = requests.get(endpoint, params=params)
        risposta.raise_for_status()  # Genera un'eccezione per risposte HTTP errate

        # Analizza la risposta JSON
        dati = risposta.json()
        risultati = dati.get("results", [])  # Estrai la lista degli utenti dalla risposta

        utenti_messaggi = {}
        for risultato in risultati:
            utente = risultato.get("username", "sconosciuto")
            numero_messaggi = risultato.get("post_count", 0)
            utenti_messaggi[utente] = numero_messaggi

        return utenti_messaggi
    except requests.exceptions.RequestException as errore:
        print(f"Errore durante il recupero dei dati: {errore}")
        print(f"Risposta completa: {errore.response.text if errore.response else 'Nessuna risposta'}")
        return {}

# Script principale da eseguire se il file viene eseguito direttamente
if __name__ == "__main__":
    utenti_messaggi = ottieni_utenti_e_messaggi_board_italiano()
    print("Utenti e numero totale di messaggi sul board italiano:")
    for utente, messaggi in utenti_messaggi.items():
        print(f"{utente}: {messaggi} messaggi")


I understand this thread is not a support group, but maybe you can point me in the right direction?

That's because you're using the wrong api url.

There is no https://api.ninjastic.space/search, it should be https://api.ninjastic.space/posts.

So endpoint should be:

Code:
endpoint = f"{BASE_URL}/posts"

I didn't look at the rest of the code since it's in italian and python which I don't have installed here for easy debugging, but I guess this is it.
legendary
Activity: 2380
Merit: 17063
Fully fledged Merit Cycler - Golden Feather 22-23
I tried to use chat GPT to use the API and I got a very bad answer:

Quote
Write state-of-the-art, well-commented code  to use ninjastic  API (documentation at https://docs.ninjastic.space )to query the database to get a list of users who wrote on the Italian board (sub-boards included) from midnight of the first day of the previous day to midnight of the first day of the current month. For example on 25 December they should query from midnight on  the first of November to the first of December at midnight

After a few correction and iterations i got this not working code.

Code:
import requests
from datetime import datetime, timedelta

# Constant for the base URL of the Ninjastic API
BASE_URL = "https://api.ninjastic.space"

# Helper function to calculate the date range
def calculate_date_range():
    """
    Calculates the date range from midnight of the first day of the previous month
    to midnight of the first day of the current month.

    Returns:
        start_date (str): The start date in ISO 8601 format.
        end_date (str): The end date in ISO 8601 format.
    """
    today = datetime.now()
    first_day_of_current_month = datetime(today.year, today.month, 1)
    first_day_of_previous_month = (first_day_of_current_month - timedelta(days=1)).replace(day=1)

    start_date = first_day_of_previous_month.isoformat()  # Removes 'Z'
    end_date = first_day_of_current_month.isoformat()     # Removes 'Z'

    return start_date, end_date

# Function to retrieve users and the number of posts from the Italian board
def get_users_and_posts_from_italian_board():
    """
    Queries the Ninjastic API to retrieve a list of users who posted on the Italian board
    (including sub-boards) and the total number of posts for each user within the specified date range.

    Returns:
        dict: A dictionary with users as keys and the number of posts as values.
    """
    start_date, end_date = calculate_date_range()

    # Endpoint and parameters for the API request
    endpoint = f"{BASE_URL}/posts"
    params = {
        "after_date": start_date,
        "before_date": end_date,
        "board": 28,  # ID of the Italian board
        "child_boards": "true"  # Include sub-boards
    }

    try:
        response = requests.get(endpoint, params=params)
        response.raise_for_status()  # Raise an exception for HTTP errors

        # Parse the JSON response
        data = response.json()
        results = data.get("results", [])  # Extract the list of users from the response

        users_posts = {}
        for result in results:
            user = result.get("username", "unknown")
            post_count = result.get("post_count", 0)
            users_posts[user] = post_count

        return users_posts
    except requests.exceptions.RequestException as error:
        print(f"Error retrieving data: {error}")
        return {}

# Main script to execute if the file is run directly
if __name__ == "__main__":
    users_posts = get_users_and_posts_from_italian_board()
    for user, posts in users_posts.items():
        print(f"{user}: {posts}")



I understand this thread is not a support group, but maybe you can point me in the right direction?


EDIT: Yes, Now I don't get any errors anymore, yet the code doesn't end anywhere
sr. member
Activity: 518
Merit: 433
Playbet.io - Crypto Casino and Sportsbook
Few hours ago I couldn't open anything on Ninjastic website, but it looks like everything is back to normal again.
This is probably some issues with servers overload or maintenance, nothing to worry about.

I confirm that everything is working here too.
Some data was lost in relation to yesterday. But maybe TryNinja will do a special scrape.
It's not working on my end
I don't know if it's from my device end.
legendary
Activity: 1890
Merit: 5204
**In BTC since 2013**
Few hours ago I couldn't open anything on Ninjastic website, but it looks like everything is back to normal again.
This is probably some issues with servers overload or maintenance, nothing to worry about.

I confirm that everything is working here too.
Some data was lost in relation to yesterday. But maybe TryNinja will do a special scrape.
legendary
Activity: 2212
Merit: 7064
Few hours ago I couldn't open anything on Ninjastic website, but it looks like everything is back to normal again.
This is probably some issues with servers overload or maintenance, nothing to worry about.
legendary
Activity: 1890
Merit: 5204
**In BTC since 2013**
Yes, I have also experienced very high response times.

Let's believe that everything will be back to normal soon.
legendary
Activity: 3696
Merit: 4343
The hacker spirit breaks any spell
Anyone else having a very long loading problem when accessing Ninjastic.space today?
I made a simple search that I usually do. At first, it went smoothly, but after a while, it took a very long time to load. After that only "Something went wrong..." results would appear.

At first, I thought it was my bad connection. But after I did a test and opened several websites. There was no problem. Is there any maintenance going on?

i have problems https://api.ninjastic.space/users/babo/ dont work and load more more and more
It's definitely going to be a problem that tryninja knows about and is working on solving so I'm not too worried

Indeed, I take this opportunity to thank him for the work he does for us
 
hero member
Activity: 826
Merit: 583
Anyone else having a very long loading problem when accessing Ninjastic.space today?
I made a simple search that I usually do. At first, it went smoothly, but after a while, it took a very long time to load. After that only "Something went wrong..." results would appear.

At first, I thought it was my bad connection. But after I did a test and opened several websites. There was no problem. Is there any maintenance going on?
legendary
Activity: 2380
Merit: 5213
I'm just wondering, the bot doesn't scrape the edited posts anymore.
Ninjastic.space scrapes every post 2 times. The first scrape is done once it sees the post and the second scrape is done 5 minutes later.
If a post is edited after 5 minutes, the edit isn't recorded by Ninjastic.space.

No edit history on Ninjastic.space means that the post has not been edited in the first 5 minutes.
hero member
Activity: 2366
Merit: 793
Bitcoin = Financial freedom
I'm just wondering, the bot doesn't scrape the edited posts anymore.

Here look at this post: https://bitcointalksearch.org/topic/--5507729

Clearly, he edited his reply after I pointed it out but when I tried the archive it showed that there were no edits found in that post, Is this something new? Or am I missing something?
hero member
Activity: 868
Merit: 530
Hey TryNinja

I believe someone requested this feature already. But I would love to request again if possible.
We all know that Symmetrick/Ratimov was a good asset of the forum before he become mad. He had some good informative threads which he deleted and fortunately it's still available on your archive. But not everyone knows how to find out them and I was looking for one of his thread which becomes impossible now. If I am not wrong, NotATether once re-created a thread with the information from one of ratimov's thread.

So, I am asking if it's possible for you to implement a system to copy the entire post with BBcode, that would be great.
legendary
Activity: 2758
Merit: 6830
I know that you scrap all the new posts that you store in a DB and then you do your little magic. For my need I would like to know (or easily see) the number of deleted posts for a user over a certain period (1 month or more if possible).
PM me, I might be able to give you a list of post ids. Can't do that on the archive itself since it's not technically viable right now.
staff
Activity: 2408
Merit: 2021
I find your lack of faith in Bitcoin disturbing.
Hi TryNinja, I'm making this post because I have a small "need" and you might be able to help me. Sorry if this has already been suggested.

I know that you scrap all the new posts that you store in a DB and then you do your little magic. For my need I would like to know (or easily see) the number of deleted posts for a user over a certain period (1 month or more if possible).

For example, in the posts section, I'd put an author, a date range and there will be an additional option that would make it possible to identify deleted posts by displaying them in red. But to do that you would have to check for each result whether the message still exists on Bitcointalk. Is this possible or too complicated ?

(I also tag LoyceV because he offers the same type of service (https://loyce.club/archive/posts/), but I have to check all the posts manually.)
legendary
Activity: 2380
Merit: 17063
Fully fledged Merit Cycler - Golden Feather 22-23
A few hours ago, I used the website to perform my usual analysis of the Italian board.
The website responded adequately, but I got a weird result: a user didn’t appear in the TOP50 chart of the top poster on the Italian board, which made me notice it.
So different result was produced.
Wondering if it could be a server issue, or the user (that would be me) did something stupid when setting up the search.
legendary
Activity: 2758
Merit: 6830
copper member
Activity: 2198
Merit: 1837
🌀 Cosmic Casino
Maybe some members had issues because of the full scan that was announced by TryNinja few days ago.
Probably.

Everything works fine on my side too

I am also noticing server issues with other websites, don't know if that is connected or not.
I actually got a "500 Internal Server Error" when trying to access talkimg.com after some slow loading time around about the same period, though I did not report it. There could be a connection. Right now, talkimg.com loads so fast as compared to what I experienced in the afternoon.
legendary
Activity: 2296
Merit: 2892
#SWGT CERTIK Audited
Everything is working fine on my side, and ninjastic.space is actually opening pages much faster than before, even after I did some stress testing.
Its working now fine for me as well, but it wasn't a couple of hours ago.
-snip-

A few hours ago, I also got the same thing, unable to connect to ninjastic.space. There was no information like "under maintenance", so I thought the site was down for something else. But earlier, I tried to reopen https://ninjastic.space/ everything was working normally again.
legendary
Activity: 1722
Merit: 5937
Everything is working fine on my side, and ninjastic.space is actually opening pages much faster than before, even after I did some stress testing.
Its working now fine for me as well, but it wasn't a couple of hours ago.

Anyway, this is a good reminder that I should already start with that local board overview so I can actually publish it in the more reasonable timeframe for a change.  Tongue

legendary
Activity: 2212
Merit: 7064
Is anyone else experiencing this? When I try to visit the site. It appears to be loading, then goes blank. I am just posting the GIF here in case it's some unexpected anomaly but if it's part of the effects of the ongoing site maintenance, then ignore  Wink
Everything is working fine on my side, and ninjastic.space is actually opening pages much faster than before, even after I did some stress testing.
Maybe some members had issues because of the full scan that was announced by TryNinja few days ago.
I am also noticing server issues with other websites, don't know if that is connected or not.
Pages:
Jump to: