Git Product home page Git Product logo

meteor-sitemaps's People

Contributors

dandv avatar fadomire avatar gadicc avatar nlhuykhang avatar picsoung avatar sachag avatar tarang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

meteor-sitemaps's Issues

Sitemaps using flowrouter and React

Does your sitemap package require Blaze templates and iron router? I tried to use it and had no errors whatsoever, but could not find where it placed the sitemap.xml file. I used the following first few lines to start the process, as you have in your documentation
sitemaps.add('/sitemap.xml', function() {

Maybe I just am not sure where to look for the file.

Thanks

sitemap index

How to create "sitemap index" by using this package or any of meteor package..??

sitemap error

Hello,

are your sitemaps working correctly? I just submitted mine generated using this package but I got a google error:

Your Sitemap appears to be an HTML page. Please use a supported sitemap format instead.

Timeout when generating bigger sitemap

Is this suitable to use with fairly bigger sitemap - 50,000 results? Recently I started to get 502 error when trying to access .xml file for this sitemap.

FATAL ERROR: Evacuation Allocation failed - process out of memory [192.81.217.53] error: Forever detected script was killed by signal: SIGABRT [192.81.217.53] error: Script restart attempt #11[192.81.217.53]

Dynamic Sitemaps

How to do dynamic sitemaps when I have something like this with Iron router:

this.route('/analytics/:_id/:name', {
   template: 'analyticsDetails',
    waitOn: function() {
        return Meteor.subscribe('Analytics', {
            _id: this.params._id
        })
    },
    data: function() {
        return Analytics.findOne(this.params._id);
    },
    onAfterAction: function () {
        SEO.set({
            title: this.params.name + " | GrowthTools.io",
            description: this.params.functionality
        });
    }
})

how to import with meteor 1.3 ?

how to import with meteor 1.3 ?
i tried import 'meteor/gadicohen:sitemaps';

and error:

ReferenceError: sitemaps is not defined

Google Webmaster Tools giving file format error

Hi!

I just tested my sitemap in Webmaster Tools and it gave this error:

Unsupported file format: Your Sitemap does not appear to be in a supported format. Please ensure it meets our Sitemap guidelines and resubmit.

I just have a single sitemaps.add statement that adds all my publicly-accessible routes, and the generated file looks correct to me (you can see it here: https://modtest.canlio.com/sitemap.xml).

Any ideas on what I'm doing wrong?

after blaze update got error for dynamic routes

hello @gadicc

updated my project to meteor 0.8. I appear to be having some issues with this sitemaps package.

I use the following code in my sitemap code to generate dynamic post for indexing:

var stories = Stories.find().fetch();
  _.each(stories, function(story){
    out.push({
      page: '/story/'+story._id+'/'+encodeURIComponent(story.title),
      lastmod: new Date().getTime(),
      changefreq: 'daily',
      priority: 0.6
    })
  });

But then I got this error server side:

W20140407-15:08:41.372(8)? (STDERR) Error: Can't wait without a fiber
W20140407-15:08:41.372(8)? (STDERR)     at Function.wait (/Users/shawnlim/.meteor/tools/c2a0453c51/lib/node_modules/fibers/future.js:83:9)
W20140407-15:08:41.372(8)? (STDERR)     at Object.Future.wait (/Users/shawnlim/.meteor/tools/c2a0453c51/lib/node_modules/fibers/future.js:325:10)
W20140407-15:08:41.372(8)? (STDERR)     at _.extend._nextObject (packages/mongo-livedata/mongo_driver.js:805)
W20140407-15:08:41.372(8)? (STDERR)     at _.extend.forEach (packages/mongo-livedata/mongo_driver.js:836)
W20140407-15:08:41.372(8)? (STDERR)     at _.extend.map (packages/mongo-livedata/mongo_driver.js:846)
W20140407-15:08:41.372(8)? (STDERR)     at _.extend.fetch (packages/mongo-livedata/mongo_driver.js:870)
W20140407-15:08:41.373(8)? (STDERR)     at Cursor.(anonymous function) [as fetch] (packages/mongo-livedata/mongo_driver.js:695)
W20140407-15:08:41.373(8)? (STDERR)     at app/server/sitemaps.js:7:32
W20140407-15:08:41.373(8)? (STDERR)     at Object.sitemaps.add.root [as handle] (packages/sitemaps/sitemaps.js:37)
W20140407-15:08:41.373(8)? (STDERR)     at next (/Users/shawnlim/.meteor/packages/webapp/9dd975a6fc/npm/node_modules/connect/lib/proto.js:190:15)

Maybe meteor adjusted a dependency you were using?

First two values returned from collection ignored

Hi, I'm trying to generate a sitemap. I have a collection "Quicks" that contains about a handful of search shortcuts. Each document of "Quicks" is rendered as an URL in the sitemap. For each I'd like to get the date of the last upload (to the collection "EboyPix") โ€” and use it as the lastmod property for the sitemap.

When iterating through "Quicks" the first two return undefined for the createdAt field. As if function "getLatestQuickDate" needed a bit of time to get going.

What can I do? Thanks!

/server/sitemap.js

// Meteor
import { sitemaps } from 'meteor/gadicohen:sitemaps';

// Collections
import { Quicks } from '../imports/api/quicks/quicks.js'; // contains search shortcuts
import { EboyPix } from '../imports/api/eboypix/eboypix.js';

const quicksLabels = Quicks.find().fetch();

// Returns the latest date from the slug
function getLatestQuickDate(slug) {
  const latestSlug = EboyPix.findOne(
    { tags: slug },
    { sort: { createdAt: -1, limit: 1 } }
  );
  if (latestSlug) {
    return latestSlug.createdAt;
  }
}

// Returns array of pages from Quicks collection
const quicksLinks = function() {
  let quicksPagesAll = [];
  if (quicksLabels) {
    Object.keys(quicksLabels).forEach(function(key) {
      const slug = quicksLabels[key].slug.toString();
      const lastmod = getLatestQuickDate(slug);
      const quicksPage = {
        page: 'pool/' + quicksLabels[key].label + '/1',
        changefreq: 'weekly',
        priority: quicksLabels[key].rank,
        lastmod: lastmod
      };
      quicksPagesAll.push(quicksPage);
    });
    return quicksPagesAll;
  }
};

sitemaps.add('/sitemap.xml', function() {
  return quicksLinks();
});

output: /sitemap.xml

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>http://localhost:3000/pool/everything/1</loc>
    <changefreq>weekly</changefreq>
    <priority>1</priority>
  </url>
  <url>
    <loc>http://localhost:3000/pool/pixorama/1</loc>
    <changefreq>weekly</changefreq>
    <priority>0.9</priority>
  </url>
  <url>
    <loc>http://localhost:3000/pool/animations/1</loc>
    <lastmod>2017-02-24T08:22:37+00:00</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.1</priority>
  </url>
  <url>
    <loc>http://localhost:3000/pool/vehicles/1</loc>
    <lastmod>2017-02-02T08:53:21+00:00</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.5</priority>
  </url>
  <url>
    <loc>http://localhost:3000/pool/portraits/1</loc>
    <lastmod>2017-02-02T21:57:44+00:00</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.7</priority>
  </url>
</urlset>

sitemap.xml.gz seems not created

Hi there!
I'm trying to apply sitemaps package functionality using next code within /server/sitemaps.jsx file:

sitemaps.config('gzip', true);
sitemaps.add('/sitemap.xml.gz', function(req) {
  console.log('Sitemap created')
  const out = [] 
  const ads = Ads.find().fetch()
  ads.forEach(ad => {
    out.push({
      ad: ad.url,
      lastmod: ad.created
    });
  });
  return out;
});

But I see that console.log on the 3d line doesn't appear. Does that mean that a gz file is not created and thus callback is not called?
What could be a problem? Permissions?
Thanks!

robots.txt smart package required ?

Hey

I try your package but it seems not to be work. This is my file:

server/sitemaps.js

console.log('Adding to Sitemap');
sitemaps.add('/sitemap.xml', function() {
  return [
    { page: '/user/login', lastmod: new Date() , changefreq: 'monthly', priority: 0.8}
  ];
});

This do .... nothing. Absolutly nothing. My console.log is outputted in my server log, but no sitemap.xml was created. I tried to add a file by my self - a sitemap.xml file - but this will also not be updated with this package.

I read, that you add a line to robots.txt when using robots.txt package - but I do not use this and I dont want to - so do I need to use this package so your will work ?

Error with spiderable

CPU usage 100% with this error on DigitalOcean

    at errnoException (child_process.js:1011:11)
    at ChildProcess.spawn (child_process.js:958:11)
    at exports.spawn (child_process.js:746:9)
    at Object.exports.execFile (child_process.js:628:15)
    at Object.Package [as handle] (packages/ongoworks_spiderable/packages/ongoworks_spiderable.js:141:1)
    at next (/opt/meteor/app/programs/server/npm/webapp/node_modules/connect/lib/proto.js:190:15)
    at packages/gadicohen_sitemaps/packages/gadicohen_sitemaps.js:71:1
Error: spawn ENOMEM
    at errnoException (child_process.js:1011:11)
    at ChildProcess.spawn (child_process.js:958:11)
    at exports.spawn (child_process.js:746:9)
    at Object.exports.execFile (child_process.js:628:15)
    at Object.Package [as handle] (packages/ongoworks_spiderable/packages/ongoworks_spiderable.js:141:1)
    at next (/opt/meteor/app/programs/server/npm/webapp/node_modules/connect/lib/proto.js:190:15)
    at packages/gadicohen_sitemaps/packages/gadicohen_sitemaps.js:71:1
Error: spawn ENOMEM
    at errnoException (child_process.js:1011:11)
    at ChildProcess.spawn (child_process.js:9[IP] 58:11)
    at exports.spawn (child_process.js:746:9)
    at Object.exports.execFile (child_process.js:628:15)
    at Object.Package [as handle] (packages/ongoworks_spiderable/packages/ongoworks_spiderable.js:141:1)
    at next (/opt/meteor/app/programs/server/npm/webapp/node_modules/connect/lib/proto.js:190:15)
    at packages/gadicohen_sitemaps/packages/gadicohen_sitemaps.js:71:1
spiderable: phantomjs failed: { [Error: Command failed: ] killed: true, code: null, signal: 'SIGTERM' } 

Sitemap is html

Hi, Thanks for this wonderful and helpful package. Works well.
There is this issue i get grom Google's webmaster's console. It says

Your Sitemap appears to be an HTML page. Please use a supported sitemap format instead.
though the sitemap's url is /sitemap.xml here's the url naijapoll.com/sitemap.xml

Path-to-RegExp for sitemap url

We create an sitemap indexes dynamically for companies by countries.

<sitemap><loc>http://example.com/sitemaps/SG/companies.xml<loc></sitemap>
<sitemap><loc>http://example.com/sitemaps/US/companies.xml<loc></sitemap>
<sitemap><loc>http://example.com/sitemaps/MY/companies.xml<loc></sitemap>
                            ...

So is it possible to implement Path-to-RegExp to sitemaps url? example:

sitemaps.add("/sitemaps/:country/companies.xml", function(params){
    var paths = []
    Company.find({'iso3':params.country}).forEach(function(doc){
        paths.push({
            page: "/company/" + doc.slug
        })
    })
    return paths;
})

Cannot read property 'find' of undefined

Yop,

I would like add your module on my Meteor App, but /sitemap.xml return this error :

TypeError: Cannot read property 'find' of undefined
   at imports/startup/server/sitemaps.js:10:17
   at packages/gadicohen_sitemaps.js:120:15 

I use latest Meteor version (5).

My sitemaps.js (server) :

import { Meteor } from 'meteor/meteor';
import { sitemaps } from 'meteor/gadicohen:sitemaps';
import { Post } from '../../api/Post.js';

sitemaps.config('rootUrl', 'https://stephane-richin.fr/');

sitemaps.add('/sitemap.xml', () => {

  const postsUrls = [];
  const posts = Post.find({ 'draft': false }).fetch();

  _.each(posts, function (page) {

    postsUrls.push({
      page: `/blog/${page.slug}`,
      lastmod: new Date()
    });

  });

  const staticPages = [
    { page: '/', lastmod: new Date() },
    { page: '/blog', lastmod: new Date() },
    { page: '/a-propos', lastmod: new Date() },
    { page: '/contact', lastmod: new Date() }
  ];

  const urls = [...staticPages, ...postsUrls];

  // required: page
  // optional: lastmod, changefreq, priority, xhtmlLinks, images, videos
  return urls;

});

Do you have any idea ?

Thank you :)

package support

@gadicc Hello, Does your package support the new version of meteor?
And is it up to date with the Google update?
If you no longer support this package. Please let users know

Adding alternate urls seems to break the XML rendering in the browser.

Hello and thank you for this package.

I noticed something interesting while I was playing with it. When we add alternate urls, the rendering of the XML document in the browser breaks.

Normal Behavior. (without alternate urls)

sitemaps.add '/sitemap.xml', ->
  return [
    { page: '/', changefreq: 'weekly' }
    { page: '/manifesto', changefreq: 'weekly' }
    { page: '/contact', changefreq: 'weekly' }
  ]

screen shot 2014-06-05 at 19 35 16

Broken Rendering. (with alternate urls)

sitemaps.add '/sitemap.xml', ->
  return [
    { page: '/', changefreq: 'weekly', xhtmlLinks: [
      { rel: 'alternate', hreflang: 'en', href: '/en' },
      { rel: 'alternate', hreflang: 'fr', href: '/fr' },
      { rel: 'alternate', hreflang: 'es', href: '/es' }
    ]}
    { page: '/manifesto', changefreq: 'weekly' }
    { page: '/contact', changefreq: 'weekly' }
  ]

screen shot 2014-06-05 at 19 35 27

# It seems that adding the following lines break the rendering process.
<url>
    <loc>http://localhost:3000/</loc>
    <changefreq>weekly</changefreq>
    <xhtml:link rel="alternate" hreflang="en" href="http://localhost:3000/en"/>
    <xhtml:link rel="alternate" hreflang="fr" href="http://localhost:3000/fr"/>
    <xhtml:link rel="alternate" hreflang="es" href="http://localhost:3000/es"/>
</url>

# but this is perfectly rendering.
<url>
    <loc>http://localhost:3000/</loc>
    <changefreq>weekly</changefreq>
</url>

When does sitemaps.add() run? Is it once, at server restart, or when?

We're wondering when the sitemap is actually created, because according to the docs, it doesn't appear that we manually call it anywhere, it just runs at some time. When does it run?

If it does run more than one time, it should duplicate the sitemap but the duplicate will have the updated collection info, which is something we need. Our collection contains ~17,000 companies and they change over time so we would like our sitemap to be updated periodically, say about once a month.

Also, somewhat related is, if it runs on some regular basis, does it delete and recreate the sitemap each time or will we need to manually remove it every time it runs?

Dynamic update sitemap.xml when DB collection updated

Is sitemap change, if db collection changed (which use to sitemap)?
For example, is sitemap update, if WikiPages was updated?

sitemaps.add('/mw_AllPages_sitemap.xml.gz', function(req) { var out = [], pages = WikiPages.find().fetch(); _.each(pages, function(page) { out.push({ page: 'read/' + page.name, lastmod: page.lastUpdated }); }); return out; });

gzip

I'd love to help implement gzip (see here). How can we proceed?

Recent Iron router #! change causes redirect loop

Navigating to /sitemap.xml results in a redirect loop between /#!sitemap.xml and /sitemap.xml.

Not sure if this should be considered an iron:router bug or a sitemaps bug.

Any thoughts, on what to check or where the issue may be?

XML Parsing Error: not well-formed

Hi,
First thanks for your awesome package.
I try to use your package but my sitemap.xml shown this error in browser:

XML Parsing Error: not well-formed
Location: http://localhost:3000/sitemap.xml
Line Number 22, Column 60:    <loc>http://localhost:3000/pages/title?q=query&lang=</loc>
---------------------------------------------------------------------------------^

But my sitemap seems to work. When viewing the source of my sitemap, the XML needed for crawlers is displayed.

URL is: http://localhost:3000/pages/title?q=query&lang=

How to add escape special character like this.

Possible to use iron router's pathFor?

Hi
I recently tried to use the pathFor in a sitemap function with a trick like this:

var pathFor = UI._globalHelpers.pathFor;
out.push({
        page: pathFor('home'),
        lastmod: new Date(2015, 0, 1),
        changefreq: 'daily',
        priority: 0.5
});

But i get the error in console saying

(STDERR) pathFor couldn't find a route named undefined

Is there a proper way for this?

Customize root URL

Currently, sitemaps' URLs are all based on Meteor.absoluteUrl(), which in turn draws from the ROOT_URL environment variable. Over at TelescopeJS/Telescope#891, we're trying to allow sites' root URLs to be user-configurable (using mongo-stored settings), so we need a way to set the root URL that doesn't involve changing environment variables.

Would you accept a PR to take a third argument on sitemaps.add which optionally specifies the root URL?

Question: How to use for simple translated page

Hello (again) Gadi!

Looks like I'm bumping into your really handy packages this week :-)

I've got a simple site for my wife's bed and breakfast which uses your meteor-headers package to figure out the language and than redirects to the language specific section:

/ -> /en and in case of Dutch / -> /nl

As such, the home is never really used since there's always the redirect to /lang

More languages will follow.

What do you think of this approach?

I'd like to configure this in the sitemap but being a n00b here maybe you can point me in the right direction?

Kind thanks,

. Dirk

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.